Categories: Crystal Growth

Crystal Growth Simulation on Linux

Crystal Growth Simulation – Part 1 walked through setting up a mesh for running crystal simulation in Elmer on Windows. Crystal Growth Simulation Failure described the process of trying to run the actual simulation on Windows and the resulting segmentation fault. What happens if we try this same experiment on Linux? First install Elmer using the instructions on the Elmer FEM blog.

sudo apt-add-repository ppa:elmer-csc-ubuntu/elmer-csc-ppa
sudo apt-get update
sudo apt-get install elmerfem-csc

I’m not sure if it’s because I’ve built other repos on my Ubuntu VM, but python3 is already installed. I still need to install pip though.

sudo apt-get install python3
sudo apt install python3-pip

pip install pyelmer
pip install objectgmsh

We need to clone 2 repos to execute the crystal growth experiment. The opencgs repo is a dependency of the test-cz-induction repo.

cd ~/repos/fem/elmer/research
git clone https://github.com/nemocrys/opencgs
git clone https://github.com/nemocrys/test-cz-induction

Next, install the opencgs module using pip then set up the crystal growth simulation mesh.

cd opencgs
pip install -e .

This installation output contains an error about an incompatible numpy version but ends with a message about all the components being successfully installed.

Requirement already satisfied: kiwisolver>=1.0.1 in /home/saint/.local/lib/python3.8/site-packages (from matplotlib->pyelmer->opencgs==0.3.1) (1.4.4)
Requirement already satisfied: fonttools>=4.22.0 in /home/saint/.local/lib/python3.8/site-packages (from matplotlib->pyelmer->opencgs==0.3.1) (4.34.4)
ERROR: pandas 1.4.3 has requirement numpy>=1.18.5; platform_machine != "aarch64" and platform_machine != "arm64" and python_version < "3.10", but you'll have numpy 1.17.4 which is incompatible.
...
Successfully installed commonmark-0.9.1 meshio-5.3.4 opencgs pandas-1.4.3 pygments-2.12.0 python-dateutil-2.8.2 pytz-2022.1 rich-12.5.1 typing-extensions-4.3.0

I ignore the error and forge ahead with running the crystal simulation setup:

cd ../test-cz-induction
python3 setup.py

This fails with an error related to numpy.

python3 setup.py 
Traceback (most recent call last):
  File "setup.py", line 19, in <module>
    import opencgs.control as ctrl
  File "/home/saint/repos/fem/elmer/research/opencgs/opencgs/__init__.py", line 5, in <module>
    import opencgs.post
  File "/home/saint/repos/fem/elmer/research/opencgs/opencgs/post.py", line 2, in <module>
    import meshio
  File "/home/saint/.local/lib/python3.8/site-packages/meshio/__init__.py", line 1, in <module>
    from . import (
  File "/home/saint/.local/lib/python3.8/site-packages/meshio/_cli/__init__.py", line 1, in <module>
    from ._main import main
  File "/home/saint/.local/lib/python3.8/site-packages/meshio/_cli/_main.py", line 5, in <module>
    from . import _ascii, _binary, _compress, _convert, _decompress, _info
  File "/home/saint/.local/lib/python3.8/site-packages/meshio/_cli/_ascii.py", line 4, in <module>
    from .. import ansys, flac3d, gmsh, mdpa, ply, stl, vtk, vtu, xdmf
  File "/home/saint/.local/lib/python3.8/site-packages/meshio/ansys/__init__.py", line 1, in <module>
    from ._ansys import read, write
  File "/home/saint/.local/lib/python3.8/site-packages/meshio/ansys/_ansys.py", line 14, in <module>
    from .._helpers import register_format
  File "/home/saint/.local/lib/python3.8/site-packages/meshio/_helpers.py", line 7, in <module>
    from numpy.typing import ArrayLike

I guess I need to heed that error from pip. python – How can I upgrade NumPy? – Stack Overflow says to use the –upgrade flag.

$ pip install numpy --upgrade
Collecting numpy
  Downloading numpy-1.23.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.1 MB)
     |████████████████████████████████| 17.1 MB 3.6 MB/s 
Installing collected packages: numpy
  WARNING: The scripts f2py, f2py3 and f2py3.8 are installed in '/home/saint/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed numpy-1.23.1

That’s all that’s needed to get things to run. For example, setting up the mesh now works:

python3 setup.py

The crystal growth simulation can now be executed as well:

python3 run.py

Interestingly, the simulation completes successfully!

crucible
melt
crystal
inductor
seed
insulation
crucible_adapter
axis_bt
...
using material graphite-CZ3R6300 from self.materials_dict
using material insulation from self.materials_dict
using material steel-1.4541 from self.materials_dict
using material vacuum from self.materials_dict
Wrote sif-file.
Starting simulation  ./simdata/2022-07-31_13-35_ss_test-cz-induction_vacuum  ...
[] [] {'CPU-time': 55.87, 'real-time': 57.49}
Finished simulation  ./simdata/2022-07-31_13-35_ss_test-cz-induction_vacuum  .
Post processing...
evaluating heat fluxes

Finished post processing.

This implies that the segmentation fault I ran into on Windows was specific to the Windows Elmer build.

Visualization

Use ParaView to visualize the results of the simulation.

sudo apt install paraview

There are different variables that can be visualized. I selected temperature for the screenshot below. Other options included joule field, joule heating, potential im, newy, etc.

Visualizing the Crystal Growth Simulation

Categories: Elmer, Simulation

Elmer Parallel Demo

Below is a video from the Elmer folks demonstrating how to use parallelization to improve the performance of Elmer. I followed along on Windows with the publicly downloadable Elmer installation. I’m listing the commands in this post for easy reference.

We first clone the demo repository then open the geo file that serves as the basis for the demo in gmsh. I ran these commands in a Windows command prompt and used full paths to the gmsh and elmer executables.

git clone https://github.com/tzwinger/elmer_parallel_demo
cd elmer_parallel_demo

# Open the geo file in gmsh as a background job
gmsh.exe elmer_flow.geo &

# Create elmer_flow.msh
gmsh.exe -3 elmer_flow.geo

ElmerGrid 14 2 elmer_flow.msh -autoclean

The ElmerGrid command is used to create partitions from the mesh input file. Running it without any arguments displays the available options. The options used in the example are highlighted below.

This program can create simple 2D structured meshes consisting of
linear, quadratic or cubic rectangles or triangles. The meshes may
also be extruded and revolved to create 3D forms. In addition many
mesh formats may be imported into Elmer software. Some options have
not been properly tested. Contact the author if you face problems.

The program has two operation modes
A) Command file mode which has the command file as the only argument
   'ElmerGrid commandfile.eg'

B) Inline mode which expects at least three input parameters
   'ElmerGrid 1 3 test'

The first parameter defines the input file format:
1)  .grd      : ElmerGrid file format
2)  .mesh.*   : Elmer input format
3)  .ep       : Elmer output format
4)  .ansys    : Ansys input format
5)  .inp      : Abaqus input format by Ideas
6)  .fil      : Abaqus output format
7)  .FDNEUT   : Gambit (Fidap) neutral file
8)  .unv      : Universal mesh file format
9)  .mphtxt   : Comsol Multiphysics mesh format
10) .dat      : Fieldview format
11) .node,.ele: Triangle 2D mesh format
12) .mesh     : Medit mesh format
13) .msh      : GID mesh format
14) .msh      : Gmsh mesh format
15) .ep.i     : Partitioned ElmerPost format
16) .2dm      : 2D triangular FVCOM format

The second parameter defines the output file format:
1)  .grd      : ElmerGrid file format
2)  .mesh.*   : ElmerSolver format (also partitioned .part format)
3)  .ep       : ElmerPost format
4)  .msh      : Gmsh mesh format
5)  .vtu      : VTK ascii XML format

The third parameter is the name of the input file.
If the file does not exist, an example with the same name is created.
The default output file name is the same with a different suffix.

There are several additional in-line parameters that are
taken into account only when applicable to the given format.
-out str             : name of the output file
-in str              : name of a secondary input file
-decimals            : number of decimals in the saved mesh (eg. 8)
...
-removeintbcs        : remove internal boundaries if they are not needed
-removelowdim        : remove boundaries that are two ranks lower than highest dim
-removeunused        : remove nodes that are not used in any element
-bulkorder           : renumber materials types from 1 so that every number is used
-boundorder          : renumber boundary types from 1 so that every number is used
-autoclean           : this performs the united action of the four above
...

Keywords are related to mesh partitioning for parallel ElmerSolver runs:
...
-metiskway int       : mesh will be partitioned with Metis using graph Kway routine

-metisrec int        : mesh will be partitioned with Metis using graph Recursive routine
-metiscontig         : enforce that the metis partitions are contiguous
-metisseed int       : random number generator seed for Metis algorithms
-partdual            : use the dual graph in partition method (when available)
...

Now the solver can be invoked. The demo first does a serial run using the ElmerSolver command.

ElmerSolver elmer_flow_gcr.sif

This took just over 6 minutes on my machine. The wall clock time is the 2nd time (highlighted below) according to the demo video.

...
MAIN: Version: 9.0 (Rev: Release, Compiled: 2020-11-10)
MAIN:  Running one task without MPI parallelization.
MAIN:  Running with just one thread per task.
MAIN:  Lua interpreted linked in.
...
ResultOutputSolver: Saving in unstructured VTK XML (.vtu) format
ResultOutputSolver: -------------------------------------
ElmerSolver: *** Elmer Solver: ALL DONE ***
ElmerSolver: The end
SOLVER TOTAL TIME(CPU,REAL):       369.31      369.31
ELMER SOLVER FINISHED AT: 2022/07/27 01:23:45

To increase parallelism, the demo continues with the serial mesh but uses OpenMP in multithreading mode by setting the OMP_NUM_THREADS environment variable. This does not appear to be sufficient to increase the number of threads it uses in my Windows setup so I need to get to the bottom of why OMP_NUM_THREADS is not being respected.

set OMP_NUM_THREADS=4
# setx /M OMP_NUM_THREADS 4
ElmerSolver elmer_flow_gcr.sif

Next, the demo uses ElmerGrid to partition the input, this time using the arguments 2 2 instead of 14 2 since we already have an existing serial Elmer mesh. It was interesting learning about partitioning algorithms like METIS – KarypisLab/METIS: METIS – Serial Graph Partitioning and Fill-reducing Matrix Ordering (github.com).

ElmerGrid 2 2 elmer_flow -partdual -metiskway 4

# notice the partitioning.4 directory in elmer_flow
dir elmer_flow

The demo uses mpirun but the Windows equivalent is “C:\Program Files\Microsoft MPI\Bin\mpiexec.exe“.

"C:\Program Files\Microsoft MPI\Bin\mpiexec.exe" -np 4 "C:\Program Files\Elmer 9.0-Release\bin\ElmerSolver.exe" elmer_flow_gcr.sif

Using 4 processes on my box takes 150s and increasing the processes to 6 drops the time to 121s.

MAIN: Version: 9.0 (Rev: Release, Compiled: 2020-11-10)
MAIN:  Running in parallel using 4 tasks.
MAIN:  Running with just one thread per task.
...
ResultOutputSolver: Saving in unstructured VTK XML (.vtu) format
ResultOutputSolver: -------------------------------------
ElmerSolver: *** Elmer Solver: ALL DONE ***
ElmerSolver: The end
SOLVER TOTAL TIME(CPU,REAL):       148.65      148.65

Visualization

The demo does not explicitly discuss how to visualize the results. However, this was covered in the Parallel Computing with Elmer talk.

Parallel Computing with Elmer

ParaView needs to be installed. Looks like they need to fix their installation dialog to not cut off text – or better yet, replace that long string with a user-friendly one. To visualize the ElmerSolver results, launch ParaView then open the parallel vtu file (*.pvtu).

Opening the Parallel VTU File

When the file opens, click on the green Apply button in the Properties panel. Notice that the pressure variable is preselected in the dropdown on the toolbar.

Visualizing Pressure in ParaView

Switching to the velocity variable shows all all blue rendering due to the 0 velocity. To see the animated visualization:

  1. Click on the slice button (4th from left in screenshot).
  2. Click on “Z Normal” in the Plane Parameters section on the Properties pane.
  3. Click on Apply.
  4. Click on the play button on the toolbar to see the animation of the flow.

The last part of the demo shows how to view the partitions in the mesh:

  1. Deselect the eye on Slice1 in the Pipeline Browser.
  2. Select the eye on “flow_gcr_t00” in the Pipeline Browser.
  3. Go to the Filter menu > Alphabetic > Connectivity.
  4. Click on the Apply button.

Change the selection in the combobox in the representation toolbar from from Surface to Surface with Edges to see how the partition seems to be based on the number of elements involved.

View of Surface with Edges

ParaView’s source code is available on GitLab so naturally, I’ll try to build it one of these fine days.


Categories: Crystal Growth, Elmer

Crystal Growth Simulation Failure

After completing the mesh generation in Crystal Growth Simulation – Part 1, I decided to investigate how to run the actual crystal growth simulation using Elmer. On Windows, Elmer needs to either be installed into “Program Files or be present in the path to avoid this error:

$ python run.py
crucible
melt
crystal
inductor
...
using material steel-1.4541 from self.materials_dict
using material vacuum from self.materials_dict
Wrote sif-file.
Starting simulation  ./simdata/2022-07-25_13-17_ss_test-cz-induction_vacuum  ...
Traceback (most recent call last):
  File "D:\dev\repos\fem\research\test-cz-induction\run.py", line 32, in <module>
    sim.execute()
  File "d:\dev\repos\fem\research\opencgs\opencgs\sim.py", line 182, in execute
    run_elmer_grid(self.sim_dir, "case.msh")
  File "C:\Python310\lib\site-packages\pyelmer\execute.py", line 28, in run_elmer_grid
    subprocess.run(args, cwd=sim_dir, stdout=f, stderr=f)
  File "C:\Python310\lib\subprocess.py", line 501, in run
    with Popen(*popenargs, **kwargs) as process:
  File "C:\Python310\lib\subprocess.py", line 966, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "C:\Python310\lib\subprocess.py", line 1435, in _execute_child
    hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2] The system cannot find the file specified

I tried installing a locally built Elmer – see How to Build Elmer on Windows. Unfortunately, the layout created by setup defaulted to “C:\Program Files\Elmer 9.0-1b8e4f7ec” (ending with the hash of the commit I built, instead of “Release”). Worse still, it didn’t launch because of missing DLLs. I need to take a closer look at how NSIS is creating the setup executable, and also figure out how to set up the publisher so that a certificate is displayed with the UAC prompt. I ended up installing the publicly downloadable Elmer then retrying the python script.

using material steel-1.4541 from self.materials_dict
using material vacuum from self.materials_dict
Wrote sif-file.
Starting simulation  ./simdata/2022-07-25_17-23_ss_test-cz-induction_vacuum_1  ...
['ERROR:: systemc: Command exit status was 1'] [] {}
Finished simulation  ./simdata/2022-07-25_17-23_ss_test-cz-induction_vacuum_1  .
Post processing...
Traceback (most recent call last):
  File "D:\dev\repos\fem\research\test-cz-induction\run.py", line 32, in <module>
    sim.execute()
  File "d:\dev\repos\fem\research\opencgs\opencgs\sim.py", line 189, in execute
    self._postprocessing_probes()
  File "d:\dev\repos\fem\research\opencgs\opencgs\sim.py", line 139, in _postprocessing_probes
    names_data = self._read_names_file(self.sim_dir + "/results/probes.dat.names")
  File "d:\dev\repos\fem\research\opencgs\opencgs\sim.py", line 129, in _read_names_file
    with open(names_file) as f:
FileNotFoundError: [Errno 2] No such file or directory: './simdata/2022-07-25_17-23_ss_test-cz-induction_vacuum_1/02_simulation/results/probes.dat.names'

A dialog popped up asking whether I wanted to grant some elmer process access to the domain network or private, etc, and I’m not sure if I took too long to answer it but the script continued with the above failure. This looks like a bug in opencgs – why continue post processing when the simulation has failed – I filed this issue to let the author know: Do not start post-processing when simulation fails · Issue #2 · nemocrys/opencgs (github.com).

I had no idea why the simulation is failing so I used Process Explorer to see which command line was being used to invoke ElmerGrid and ElmerSolver. I was able to then manually invoke ElmerSolver and notice a segmentation fault!

ELMER SOLVER (v 9.0) STARTED AT: 2022/07/25 17:23:13
ParCommInit:  Initialize #PEs:            1
MAIN: 
MAIN: =============================================================
MAIN: ElmerSolver finite element software, Welcome!
...
MAIN: Reading Model: case.sif
 Caught LUA error:[string "loadfile("C:/Program Files (x86)/Elmer/shar..."]:1: attempt to call a nil value
 Caught LUA error:[string "loadstring(readsif("case.sif"))()"]:1: attempt to call global 'readsif' (a nil value)
LoadInputFile: Scanning input file: case.sif
LoadInputFile: Scanning only size info
...
OptimizeBandwidth: Half bandwidth after optimization: 1447
OptimizeBandwidth: ---------------------------------------------------------
'ViewFactors' is not recognized as an internal or external command,
operable program or batch file.
ERROR:: systemc: Command exit status was 1
RadiationFactors: Message
RadiationFactors: All done time (s)                      4.9000E-02
...
      60 0.2041E-08
      61 0.3710E-09
ComputeChange: NS (ITER=11) (NRM,RELC): ( 0.17032920E-04 0.41334346E-05 ) :: statmagsolver
StatMagSolve: Convergence after 11 iterations
StatMagSolve: Joule Heating (W):      4.1375E+01
StatMagSolve: All done, exiting
StatMagSolve: ------------------------------------------------
ComputeChange: SS (ITER=1) (NRM,RELC): ( 0.17032920E-04  2.0000000     ) :: statmagsolver
HeatSolve:  Found Control Point at distance:   0.0000000000000000
HeatSolve:  Control Point Index:           0
HeatSolve: Using Steady-state Heater Control
HeatSolve: 
HeatSolve: 
HeatSolve: -------------------------------------
HeatSolve:  TEMPERATURE ITERATION           1
HeatSolve: -------------------------------------
HeatSolve: 
HeatSolve: Starting Assembly...

Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Backtrace for this error:
#0  0xffffffff
#1  0xffffffff
#2  0xffffffff
...
#19  0xffffffff
#20  0xffffffff

Looking at the simulation folders such as “simdata\2022-07-25_22-52_ss_test-cz-induction_vacuum_1\02_simulation” revealed that there are ElmerGrid and ElmerSolver log files present with this info, so I didn’t even need Process Explorer! At this point, I have to save the SIGSEGV investigation for another day.


Crystal Growth Simulation – Part 1

Ever since I read chapter 2 of Fabrication Engineering at the Micro- and Nanoscale, I have been interested in simulating crystal growth, primarily because of the multi-discplinary nature of this problem: finite element modeling software, modeling using high performance computing, the physics of the problem (governing laws and equations), and visualization techniques. I had run across Elmer when looking up crystal growth examples a while back. About two weeks ago, I was watching this Elmer demo in which they used gmsh to show the geometry for the simulation.

I wanted to learn more about Gmsh and how it is used, so I took a detour into this Gmsh introductory video. It was an excellent tour of Gmsh and its features.

Gmsh for Crystal Growth

At this point, I wanted to see how Gmsh could be used for crystal growth simulation so I searched for “crystal growth gmsh“. I was very pleased to find this paper: Development and validation of a thermal simulation for the Czochralski crystal growth process using model experiments – ScienceDirect. It appeared to have a corresponding set of slides and a talk as well. It especially piqued my interest that there is a script to set up a mesh for a Czochralski furnace.

Running the Code

Thankfully, the code in this paper is open source. I already had python installed so the first thing I tried was executing run.py in Git bash

$ git clone https://github.com/nemocrys/test-cz-induction
$ cd test-cz-induction
$ python run.py
Traceback (most recent call last):
  File "D:\dev\repos\fem\research\test-cz-induction\run.py", line 5, in <module>
    import opencgs.control as ctrl
ModuleNotFoundError: No module named 'opencgs'

Without using my brain, I decided to install pyelmer since I knew it was a dependency (from the talk).

$ pip install pyelmer
Collecting pyelmer
  Downloading pyelmer-1.0.0-py3-none-any.whl (27 kB)
Collecting gmsh
  Downloading gmsh-4.10.5-py2.py3-none-win_amd64.whl (38.4 MB)
     |████████████████████████████████| 38.4 MB 285 kB/s
Collecting pyyaml
  Downloading PyYAML-6.0-cp310-cp310-win_amd64.whl (151 kB)
     |████████████████████████████████| 151 kB ...
Collecting matplotlib
  Downloading matplotlib-3.5.2-cp310-cp310-win_amd64.whl (7.2 MB)
     |████████████████████████████████| 7.2 MB 6.4 MB/s
Collecting pillow>=6.2.0
  Downloading Pillow-9.2.0-cp310-cp310-win_amd64.whl (3.3 MB)
     |████████████████████████████████| 3.3 MB ...
Collecting cycler>=0.10
  Downloading cycler-0.11.0-py3-none-any.whl (6.4 kB)
Collecting numpy>=1.17
  Downloading numpy-1.23.1-cp310-cp310-win_amd64.whl (14.6 MB)
     |████████████████████████████████| 14.6 MB 6.4 MB/s
Collecting packaging>=20.0
  Downloading packaging-21.3-py3-none-any.whl (40 kB)
     |████████████████████████████████| 40 kB 2.5 MB/s
Collecting python-dateutil>=2.7
  Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
     |████████████████████████████████| 247 kB ...
Collecting kiwisolver>=1.0.1
  Downloading kiwisolver-1.4.4-cp310-cp310-win_amd64.whl (55 kB)
     |████████████████████████████████| 55 kB 1.6 MB/s
Collecting pyparsing>=2.2.1
  Downloading pyparsing-3.0.9-py3-none-any.whl (98 kB)
     |████████████████████████████████| 98 kB 6.8 MB/s
Collecting fonttools>=4.22.0
  Downloading fonttools-4.34.4-py3-none-any.whl (944 kB)
     |████████████████████████████████| 944 kB 6.4 MB/s
Collecting six>=1.5
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Installing collected packages: six, pyparsing, python-dateutil, pillow, packaging, numpy, kiwisolver, fonttools, cycler, pyyaml, matplotlib, gmsh, pyelmer
  WARNING: Failed to write executable - trying to use .deleteme logic
ERROR: Could not install packages due to an OSError: [WinError 2] The system cannot find the file specified: 'C:\\Python310\\Scripts\\f2py.exe' -> 'C:\\Python310\\Scripts\\f2py.exe.deleteme'

WARNING: You are using pip version 21.2.4; however, version 22.2 is available.
You should consider upgrading via the 'C:\Python310\python.exe -m pip install --upgrade pip' command.

Looks like I need to run this as an administrator.

C:\dev>pip install pyelmer
Collecting pyelmer
  Using cached pyelmer-1.0.0-py3-none-any.whl (27 kB)
Collecting pyyaml
  Using cached PyYAML-6.0-cp310-cp310-win_amd64.whl (151 kB)
Collecting gmsh
  Using cached gmsh-4.10.5-py2.py3-none-win_amd64.whl (38.4 MB)
Collecting matplotlib
  Using cached matplotlib-3.5.2-cp310-cp310-win_amd64.whl (7.2 MB)
Requirement already satisfied: python-dateutil>=2.7 in c:\python310\lib\site-packages (from matplotlib->pyelmer) (2.8.2)
Requirement already satisfied: packaging>=20.0 in c:\python310\lib\site-packages (from matplotlib->pyelmer) (21.3)
Requirement already satisfied: fonttools>=4.22.0 in c:\python310\lib\site-packages (from matplotlib->pyelmer) (4.34.4)
Requirement already satisfied: kiwisolver>=1.0.1 in c:\python310\lib\site-packages (from matplotlib->pyelmer) (1.4.4)
Requirement already satisfied: pillow>=6.2.0 in c:\python310\lib\site-packages (from matplotlib->pyelmer) (9.2.0)
Requirement already satisfied: pyparsing>=2.2.1 in c:\python310\lib\site-packages (from matplotlib->pyelmer) (3.0.9)
Requirement already satisfied: numpy>=1.17 in c:\python310\lib\site-packages (from matplotlib->pyelmer) (1.23.1)
Collecting cycler>=0.10
  Using cached cycler-0.11.0-py3-none-any.whl (6.4 kB)
Requirement already satisfied: six>=1.5 in c:\python310\lib\site-packages (from python-dateutil>=2.7->matplotlib->pyelmer) (1.16.0)
Installing collected packages: cycler, pyyaml, matplotlib, gmsh, pyelmer
Successfully installed cycler-0.11.0 gmsh-4.10.5 matplotlib-3.5.2 pyelmer-1.0.0 pyyaml-6.0
WARNING: You are using pip version 21.2.4; however, version 22.2 is available.
You should consider upgrading via the 'C:\Python310\python.exe -m pip install --upgrade pip' command.

Installing pyelmer is not sufficient to avoid the “No module named ‘opencgs’” error. The solution is to clone the opencgs repo and use pip to install (as administrator) from that repo.

git clone https://github.com/nemocrys/opencgs
cd opencgs
D:\...\research\opencgs>pip install -e .
Obtaining file:///D:/dev/repos/fem/research/opencgs
Collecting meshio
  Using cached meshio-5.3.4-py3-none-any.whl (167 kB)
Collecting pandas
  Using cached pandas-1.4.3-cp310-cp310-win_amd64.whl (10.5 MB)
Requirement already satisfied: pyyaml in c:\python310\lib\site-packages (from opencgs==0.3.1) (6.0)
Requirement already satisfied: pyelmer in c:\python310\lib\site-packages (from opencgs==0.3.1) (1.0.0)
Collecting rich
  Using cached rich-12.5.1-py3-none-any.whl (235 kB)
Requirement already satisfied: numpy in c:\python310\lib\site-packages (from meshio->opencgs==0.3.1) (1.23.1)
Requirement already satisfied: python-dateutil>=2.8.1 in c:\python310\lib\site-packages (from pandas->opencgs==0.3.1) (2.8.2)
Collecting pytz>=2020.1
  Using cached pytz-2022.1-py2.py3-none-any.whl (503 kB)
Requirement already satisfied: six>=1.5 in c:\python310\lib\site-packages (from python-dateutil>=2.8.1->pandas->opencgs==0.3.1) (1.16.0)
Requirement already satisfied: matplotlib in c:\python310\lib\site-packages (from pyelmer->opencgs==0.3.1) (3.5.2)
Requirement already satisfied: gmsh in c:\python310\lib\site-packages (from pyelmer->opencgs==0.3.1) (4.10.5)
Requirement already satisfied: kiwisolver>=1.0.1 in c:\python310\lib\site-packages (from matplotlib->pyelmer->opencgs==0.3.1) (1.4.4)
Requirement already satisfied: pyparsing>=2.2.1 in c:\python310\lib\site-packages (from matplotlib->pyelmer->opencgs==0.3.1) (3.0.9)
Requirement already satisfied: cycler>=0.10 in c:\python310\lib\site-packages (from matplotlib->pyelmer->opencgs==0.3.1) (0.11.0)
Requirement already satisfied: pillow>=6.2.0 in c:\python310\lib\site-packages (from matplotlib->pyelmer->opencgs==0.3.1) (9.2.0)
Requirement already satisfied: fonttools>=4.22.0 in c:\python310\lib\site-packages (from matplotlib->pyelmer->opencgs==0.3.1) (4.34.4)
Requirement already satisfied: packaging>=20.0 in c:\python310\lib\site-packages (from matplotlib->pyelmer->opencgs==0.3.1) (21.3)
Collecting commonmark<0.10.0,>=0.9.0
  Using cached commonmark-0.9.1-py2.py3-none-any.whl (51 kB)
Requirement already satisfied: pygments<3.0.0,>=2.6.0 in c:\python310\lib\site-packages (from rich->meshio->opencgs==0.3.1) (2.12.0)
Installing collected packages: commonmark, rich, pytz, pandas, meshio, opencgs
  Running setup.py develop for opencgs
Successfully installed commonmark-0.9.1 meshio-5.3.4 opencgs-0.3.1 pandas-1.4.3 pytz-2022.1 rich-12.5.1
WARNING: You are using pip version 21.2.4; however, version 22.2 is available.
You should consider upgrading via the 'C:\Python310\python.exe -m pip install --upgrade pip' command.

Trying to run gives an error about a missing module named objectgmsh. My assumption was the installing pyelmer was supposed to bring in all such modules.

D:\dev\repos\fem\research\test-cz-induction>python run.py
Traceback (most recent call last):
  File "D:\dev\repos\fem\research\test-cz-induction\run.py", line 5, in <module>
    import opencgs.control as ctrl
  File "d:\dev\repos\fem\research\opencgs\opencgs\__init__.py", line 3, in <module>
    import opencgs.geo
  File "d:\dev\repos\fem\research\opencgs\opencgs\geo.py", line 1, in <module>
    from objectgmsh import *
ModuleNotFoundError: No module named 'objectgmsh'

I notice while reviewing the paper yet again that it mentions the setup being in a Dockerfile. Sure enough, objectgmsh is a separate package that needs to be installed using pip. I upgrade pip for good measure since those warnings are getting annoying.

\test-cz-induction>pip install objectgmsh
Collecting objectgmsh
  Downloading objectgmsh-0.9-py3-none-any.whl (25 kB)
Requirement already satisfied: gmsh in c:\python310\lib\site-packages (from objectgmsh) (4.10.5)
Requirement already satisfied: numpy in c:\python310\lib\site-packages (from objectgmsh) (1.23.1)
Installing collected packages: objectgmsh
Successfully installed objectgmsh-0.9

I realize I should probably be running setup.py instead of run.py. Here’s the output now!

\test-cz-induction>python setup.py
crucible
melt
crystal
inductor
seed
insulation
crucible_adapter
axis_bt
vessel
axis_top
crucible 0.001999999999999999
melt 0.003
crystal 0.0004
inductor 0.001
seed 0.00017999999999999998
insulation 0.005
crucible_adapter 0.0039
axis_bt 0.0025
vessel 0.0025
axis_top 0.0025
filling 0.0263
Warning: Mesh size = 0 for bnd_melt. Ignoring this shape...
Warning: Mesh size = 0 for bnd_seed. Ignoring this shape...
Warning: Mesh size = 0 for bnd_crystal_side. Ignoring this shape...
Warning: Mesh size = 0 for bnd_crystal_top. Ignoring this shape...
Warning: Mesh size = 0 for bnd_axtop_side. Ignoring this shape...
Warning: Mesh size = 0 for bnd_axtop_bt. Ignoring this shape...
Warning: Mesh size = 0 for bnd_crucible_bt. Ignoring this shape...
Warning: Mesh size = 0 for bnd_crucible_outside. Ignoring this shape...
Warning: Mesh size = 0 for bnd_ins. Ignoring this shape...
Warning: Mesh size = 0 for bnd_adp. Ignoring this shape...
Warning: Mesh size = 0 for bnd_axbt. Ignoring this shape...
Warning: Mesh size = 0 for bnd_vessel_inside. Ignoring this shape...
Warning: Mesh size = 0 for bnd_vessel_outside. Ignoring this shape...
Warning: Mesh size = 0 for bnd_inductor_outside. Ignoring this shape...
Warning: Mesh size = 0 for bnd_inductor_inside. Ignoring this shape...
Warning: Mesh size = 0 for bnd_symmetry_axis. Ignoring this shape...
Warning: Mesh size = 0 for if_crucible_melt. Ignoring this shape...
Warning: Mesh size = 0 for if_melt_crystal. Ignoring this shape...
Warning: Mesh size = 0 for if_crystal_seed. Ignoring this shape...
Warning: Mesh size = 0 for if_seed_axtop. Ignoring this shape...
Warning: Mesh size = 0 for if_axtop_vessel. Ignoring this shape...
Warning: Mesh size = 0 for if_crucible_ins. Ignoring this shape...
Warning: Mesh size = 0 for if_ins_adp. Ignoring this shape...
Warning: Mesh size = 0 for if_adp_axbt. Ignoring this shape...
Warning: Mesh size = 0 for if_axbt_vessel. Ignoring this shape...
Info    : Meshing 1D...
Info    : [  0%] Meshing curve 3 (Line)
Info    : [ 10%] Meshing curve 4 (Line)
Info    : [ 10%] Meshing curve 15 (Line)
Info    : [ 10%] Meshing curve 16 (Line)
Info    : [ 10%] Meshing curve 17 (Line)
Info    : [ 10%] Meshing curve 18 (BSpline)
Info    : [ 20%] Meshing curve 19 (Circle)
...
Info    : [100%] Meshing surface 12 order 2
Info    : [100%] Meshing surface 13 order 2
Info    : [100%] Meshing surface 14 order 2
Info    : Surface mesh: worst distortion = 0.526353 (0 elements in ]0, 0.2]); worst gamma = 0.252838
Info    : Done meshing order 2 (Wall 0.990998s, CPU 0.75s)
-------------------------------------------------------
Version       : 4.10.5
License       : GNU General Public License
Build OS      : Windows64-sdk
Build date    : 20220701
Build host    : gmsh.info
Build options : 64Bit ALGLIB[contrib] ANN[contrib] Bamg Blas[petsc] Blossom Cgns DIntegration DomHex Eigen[contrib] Fltk Gmm[contrib] Hxt Jpeg Kbipack Lapack[petsc] MathEx[contrib] Med Mesh Metis[contrib] Mmg Mpeg Netgen NoSocklenT ONELAB ONELABMetamodel OpenCASCADE OpenCASCADE-CAF OpenGL OpenMP OptHom PETSc Parser Plugins Png Post QuadMeshingTools QuadTri Solver TetGen/BR Voro++[contrib] WinslowUntangler Zlib
FLTK version  : 1.4.0
PETSc version : 3.15.0 (real arithmtic)
OCC version   : 7.6.1
MED version   : 4.1.0
Packaged by   : nt authority system
Web site      : https://gmsh.info
Issue tracker : https://gitlab.onelab.info/gmsh/gmsh/issues
-------------------------------------------------------

Gmsh launches (although not as a separate process) with the generated mesh:

Crystal Growth Gmsh Model

setup.py outputs the path to the mesh after closing Gmsh.

Info    : Writing './simdata/_test/case.msh'...
Info    : Done writing './simdata/_test/case.msh'
Gmsh model created with objectgmsh.
Shapes: ['crucible', 'melt', 'crystal', 'inductor', 'seed', 'insulation', 'crucible_adapter', 'axis_bt', 'vessel', 'axis_top', 'filling', 'bnd_melt', 'bnd_seed', 'bnd_crystal_side', 'bnd_crystal_top', 'bnd_axtop_side', 'bnd_axtop_bt', 'bnd_crucible_bt', 'bnd_crucible_outside', 'bnd_ins', 'bnd_adp', 'bnd_axbt', 'bnd_vessel_inside', 'bnd_vessel_outside', 'bnd_inductor_outside', 'bnd_inductor_inside', 'bnd_symmetry_axis', 'if_crucible_melt', 'if_melt_crystal', 'if_crystal_seed', 'if_seed_axtop', 'if_axtop_vessel', 'if_crucible_ins', 'if_ins_adp', 'if_adp_axbt', 'if_axbt_vessel']

The next step will be to figure out how to simulate crystal growth using pyelmer.


Investigating how to Build Elmer on Windows

The instructions for building the Elmer source code are really simple! I decided to try them on Windows. The Developer Command Prompt is necessary for cmake (as far as I can tell). Note that C, C++, and Fortran compilers are required for building Elmer.

cd \dev\repos
mkdir fem
git clone git://www.github.com/ElmerCSC/elmerfem 
mkdir build
cd build
cmake -DWITH_ELMERGUI:BOOL=TRUE -DWITH_MPI:BOOL=FALSE -DCMAKE_INSTALL_PREFIX=../install ../elmerfem

I discovered that a Fortran compiler is required when I got this error on my first build attempt:

-- Building for: Visual Studio 17 2022
-- The Fortran compiler identification is unknown
-- The C compiler identification is MSVC 19.32.31326.0
-- The CXX compiler identification is MSVC 19.32.31326.0
CMake Error at CMakeLists.txt:34 (PROJECT):
  No CMAKE_Fortran_COMPILER could be found.

Line 34 of CMakeLists.txtPROJECT(Elmer Fortran C CXX) – uses the PROJECT cmake command to set the project name to “Elmer” and specify the programming languages required, hence the build failure above.

Installing a Fortran Compiler – GFortran?

GFortran looks like the only free Fortran compiler out there so I grabbed the compiler from Fortran, C, C++ for Windows (equation.com) as recommended by Installing GFortran – (fortran-lang.org). The newly installed Fortran compiler was not automatically detected by CMake. Based on the discussion at c++ – Error: No CMAKE_Fortran_COMPILER could be found for Visual Studio 2019 Fortran support – Stack Overflow, I made this change to CMakeLists.txt to pick up the GFortran compiler:

--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -21,6 +21,8 @@ if(APPLE)
   # option(HUNTER_ENABLED "Enable Hunter package manager support" OFF)
   # set (CMAKE_GENERATOR "Unix Makefiles" CACHE INTERNAL "" FORCE)
   # set(CMAKE_TRY_COMPILE_TARGET_TYPE "STATIC_LIBRARY")
+else()
+  set(CMAKE_Fortran_COMPILER "C:/dev/software/gcc/bin/gfortran.exe")
 endif()

Unfortunately, that wasn’t sufficient to address the build failure. Interestingly, someone else ran into this exact same issue at windows – The MinGW gfortran compiler is not able to compile a simple test program – Stack Overflow. Sad times though when StackOverflow does not have an answer! Their solution for specifying a custom compiler is much cleaner – simply define the CMake variable when invoking cmake!

cmake -DWITH_ELMERGUI:BOOL=TRUE -DWITH_MPI:BOOL=FALSE -DCMAKE_INSTALL_PREFIX=../install -DCMAKE_Fortran_COMPILER=C:/dev/software/gcc/bin/gfortran.exe ../elmerfem

Searching for the error message “The Fortran compiler identification is unknown (bing.com)” reveals an existing GitHub issue issue Cannot build using cmake with gfortran on Windows — the Fortran compiler identification is unknown · Issue #328 · fortran-lang/stdlib. Someone mentioned that the MinGW compiler worked fine.

Installing a Fortran Compiler – MinGW

Via Cygwin

The MinGW-w64 downloads looked promising. Since I already had Cygwin installed, I installed the GFortran package. The path to the GFortran compiler can be retrieved using the Cygwin command cygpath -w `which gfortran` and passed to CMake. That still didn’t work.

Installing gcc-fortran
setup-x86_64.exe q -P gcc-fortran

cmake -DWITH_ELMERGUI:BOOL=TRUE -DWITH_MPI:BOOL=FALSE -DCMAKE_INSTALL_PREFIX=../install -DCMAKE_Fortran_COMPILER=C:/dev/cygwin64/bin/gfortran.exe ../elmerfem

At least that showed the mingw Fortran compiler package name mingw64-x86_64-gcc-fortran. Interestingly, that package is marked already installed!

Via MSYS2

Since Cygwin didn’t simply work, I decided to try installing MSYS2 (before resorting to uninstalling the Cygwin gcc-fortran package). The Fortran compiler is installed by MSYS2. Once setup completes, CMake also fails when using the MinGW Fortran compiler!

cmake -DWITH_ELMERGUI:BOOL=TRUE -DWITH_MPI:BOOL=FALSE -DCMAKE_INSTALL_PREFIX=../install -DCMAKE_Fortran_COMPILER=D:/dev/Software/msys64/mingw64/bin/gfortran.exe ../elmerfem

Debugging the Fortran Detection Failure

Since none of the compilers work, let’s take a closer look at the error:

$ cmake -DWITH_ELMERGUI:BOOL=TRUE -DWITH_MPI:BOOL=FALSE -DCMAKE_INSTALL_PREFIX=../install -DCMAKE_Fortran_COMPILER=C:/dev/software/gcc/bin/gfortran.exe ../elmerfem
-- The Fortran compiler identification is unknown
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - failed
-- Check for working Fortran compiler: C:/dev/software/gcc/bin/gfortran.exe
-- Check for working Fortran compiler: C:/dev/software/gcc/bin/gfortran.exe - broken
CMake Error at C:/Program Files/Microsoft Visual Studio/2022/Preview/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.22/Modules/CMakeTestFortranCompiler.cmake:61 (message):
  The Fortran compiler

    "C:/dev/software/gcc/bin/gfortran.exe"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: D:/dev/repos/fem/build/CMakeFiles/CMakeTmp

    Run Build Command(s):C:/Program Files/Microsoft Visual Studio/2022/Preview/Common7/IDE/devenv.com CMAKE_TRY_COMPILE.sln /build Debug /project cmTC_4528a &&
    Microsoft Visual Studio 2022 Version 17.3.0 Preview 1.0 [...].
    Copyright (C) Microsoft Corp. All rights reserved.

    The operation could not be completed. The parameter is incorrect.

    Use:
    devenv  [solutionfile | projectfile | folder | anyfile.ext]  [switches]
...

To get a sense of what could be going wrong, I opened the folder containing the temporary project CMake is trying to build. Its contents are deleted before CMake terminates. However, the build was slow enough for me to copy all the files into another temp folder to repro this failure. Running the devenv.com command above fails with the same error.

Interestingly, loading the solution in Visual Studio results in an error because one of the projects cannot be loaded! However, that project file has a .vfproj extension (which seems specific to the Intel Fortran compiler, e.g. as described at Cannot open vfproj file in visual studio 2017 – Intel Communities).

Looks like it’s the CMakeTestFortranCompiler.cmake file that is generating Intel Fortran projects. The first check that file is:

if(CMAKE_Fortran_COMPILER_FORCED)
  # The compiler configuration was forced by the user.
  # Assume the user has configured all compiler information.
  set(CMAKE_Fortran_COMPILER_WORKS TRUE)
  return()
endif()

The CMAKE_Fortran_COMPILER_FORCED define can be used to bail out of the custom configuration so define it when invoking cmake:

cmake -DWITH_ELMERGUI:BOOL=TRUE -DWITH_MPI:BOOL=FALSE -DCMAKE_INSTALL_PREFIX=../install -DCMAKE_Fortran_COMPILER=D:/dev/Software/msys64/mingw64/bin/gfortran.exe -DCMAKE_Fortran_COMPILER_FORCED:BOOL=TRUE ../elmerfem

We now get a new error! Finally making some progress!

cmake -DWITH_ELMERGUI:BOOL=TRUE -DWITH_MPI:BOOL=FALSE -DCMAKE_INSTALL_PREFIX=../install -DCMAKE_Fortran_COMPILER=D:/dev/Software/msys64/mingw64/bin/gfortran.exe -DCMAKE_Fortran_COMPILER_FORCED:BOOL=TRUE ../elmerfem
-- The Fortran compiler identification is unknown
CMake Deprecation Warning at cmake/Modules/FindMKL.cmake:2 (CMAKE_MINIMUM_REQUIRED):
  Compatibility with CMake < 2.8.12 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.
Call Stack (most recent call first):
  CMakeLists.txt:308 (FIND_PACKAGE)


-- ------------------------------------------------
-- Looking for Fortran sgemm
-- Looking for Fortran sgemm - not found
-- Looking for pthread.h
-- Looking for pthread.h - not found
-- Found Threads: TRUE
CMake Error at C:/Program Files/Microsoft Visual Studio/2022/Preview/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find BLAS (missing: BLAS_LIBRARIES)
Call Stack (most recent call first):
  C:/Program Files/Microsoft Visual Studio/2022/Preview/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
  C:/Program Files/Microsoft Visual Studio/2022/Preview/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.22/Modules/FindBLAS.cmake:1337 (find_package_handle_standard_args)
  CMakeLists.txt:433 (FIND_PACKAGE)


-- Configuring incomplete, errors occurred!
See also "D:/dev/repos/fem/build/CMakeFiles/CMakeOutput.log".
See also "D:/dev/repos/fem/build/CMakeFiles/CMakeError.log".

This error is from the FindBLAS module (see FindBLAS source code I’ve linked to in the error log above). It should be able to find BLAS as per this question Can CMake FindBLAS find OpenBLAS? – Stack Overflow.

  1. https://duckduckgo.com/?q=gfortran+blas
  2. fortran – Error in linking gfortran to LAPACK and BLAS – Stack Overflow

Installing BLAS

Searching for “pacman blas” leads to fortran – Using BLAS, LAPACK, and ARPACK with MSYS2 – Stack Overflow which points out that you can search for packages using pacman -Ss. The -S flag stands for sync. Use pacman -Sh to see the package sync options. See Package Management – MSYS2 for more details.

# Search for BLAS packages
pacman -Ss blas

# Install mingw BLAS package
pacman -S mingw64/mingw-w64-x86_64-openblas

# Install LAPACK
pacman -S mingw64/mingw-w64-x86_64-lapack

The output should look like this when complete:

$ pacman -S mingw64/mingw-w64-x86_64-openblas
resolving dependencies...
looking for conflicting packages...

Packages (1) mingw-w64-x86_64-openblas-0.3.20-3

Total Download Size:    11.76 MiB
Total Installed Size:  103.67 MiB

:: Proceed with installation? [Y/n] y
:: Retrieving packages...
 mingw-w64-x86_64-openblas-0.3.20-3-any                                                                                                 11.8 MiB  2.26 MiB/s 00:05 [#####...#####] 100%
(1/1) checking keys in keyring             [#####...#####] 100%
(1/1) checking package integrity           [#####...#####] 100%
(1/1) loading package files                [#####...#####] 100%
(1/1) checking for file conflicts          [#####...#####] 100%
(1/1) checking available disk space        [#####...#####] 100%
:: Processing package changes...
(1/1) installing mingw-w64-x86_64-openblas [#####...#####] 100%
Set the environment variable OPENBLAS_NUM_THREADS to the
number of threads to use.resolving dependencies...

This doesn’t address the errors. A search for the exact error message “Could NOT find BLAS (missing: BLAS_LIBRARIES)” reveals a useful GitHub discussion at find_package(BLAS) failed with CMake · Issue #2440 · mxe/mxe. So BLAS_LIBRARIES can simply be defined at the command line!

cmake -DWITH_ELMERGUI:BOOL=TRUE -DWITH_MPI:BOOL=FALSE -DCMAKE_INSTALL_PREFIX=../install -DCMAKE_Fortran_COMPILER=D:/dev/Software/msys64/mingw64/bin/gfortran.exe  -DBLAS_LIBRARIES=D:/dev/Software/msys64/mingw64/lib ../elmerfem

We now get a new error about LAPACK_LIBRARIES and define it as well!

cmake -DWITH_ELMERGUI:BOOL=TRUE -DWITH_MPI:BOOL=FALSE -DCMAKE_INSTALL_PREFIX=../install -DCMAKE_Fortran_COMPILER=D:/dev/Software/msys64/mingw64/bin/gfortran.exe -DCMAKE_Fortran_COMPILER_FORCED:BOOL=TRUE -DBLAS_LIBRARIES=D:/dev/Software/msys64/mingw64/lib -DLAPACK_LIBRARIES=D:/dev/Software/msys64/mingw64/lib ../elmerfem

This finally gets us past the missing package issues and on to more Fortran compiler errors!

-- Found LAPACK: D:/dev/Software/msys64/mingw64/lib
-- Checking whether D:/dev/Software/msys64/mingw64/bin/gfortran.exe supports PROCEDURE POINTER
-- Checking whether D:/dev/Software/msys64/mingw64/bin/gfortran.exe supports PROCEDURE POINTER -- no
CMake Error at CMakeLists.txt:477 (MESSAGE):
  Fortran compiler does not seem to support the PROCEDURE statement.

Support for PROCEDURE Statements

CMakeLists.txt:475 is this line INCLUDE(testProcedurePointer). The included script tests the Fortran compiler but does not explain why the test fails. To see the details, append the string : ${OUTPUT} to the end of the string “Checking whether ${CMAKE_Fortran_COMPILER} supports PROCEDURE POINTER — no” (just before the closing quote). The error message now contains additional information – the same error from earlier! Opening the solution in Visual Studio confirms that yet another unsupported .vfproj has been generated.

Change Dir: D:/dev/repos/fem/build/CMakeFiles/CMakeTmp

Run Build Command(s):C:/Program Files/Microsoft Visual Studio/2022/Preview/Common7/IDE/devenv.com CMAKE_TRY_COMPILE.sln /build Debug /project cmTC_77a33 &&
Microsoft Visual Studio 2022 Version 17.3.0 Preview 1.0 [...].
Copyright (C) Microsoft Corp. All rights reserved.

The operation could not be completed. The parameter is incorrect.

Use:
devenv  [solutionfile | projectfile | folder | anyfile.ext]  [switches]

<Updated VS, unfortunately changing the CMake version>. This is the CMakeLists.txt generated for the solution:

cmake_minimum_required(VERSION 3.22.22022201.0)
set(CMAKE_MODULE_PATH "D:/dev/repos/fem/elmerfem/cmake/Modules;C:/Program Files/Microsoft Visual Studio/2022/Preview/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.22/Modules")
cmake_policy(SET CMP0091 OLD)
cmake_policy(SET CMP0126 OLD)
project(CMAKE_TRY_COMPILE Fortran)
set(CMAKE_VERBOSE_MAKEFILE 1)
set(CMAKE_Fortran_FLAGS "")
set(CMAKE_Fortran_FLAGS "${CMAKE_Fortran_FLAGS} ${COMPILE_DEFINITIONS}")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${EXE_LINKER_FLAGS}")
include_directories(${INCLUDE_DIRECTORIES})
set(CMAKE_SUPPRESS_REGENERATION 1)
link_directories(${LINK_DIRECTORIES})
cmake_policy(SET CMP0065 OLD)
cmake_policy(SET CMP0083 OLD)
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY "D:/dev/repos/fem/build/CMakeFiles/CMakeTmp")
add_executable(cmTC_b909d "D:/dev/repos/fem/build/CMakeFiles/CMakeTmp/testFortranProcedurePointer.f90")
target_link_libraries(cmTC_b909d ${LINK_LIBRARIES})

The cmake project command names the generated project CMAKE_TRY_COMPILE and specifies that the Fortran programming language is needed to build the project. At this point, it looks like a question of how the project is generated. Searching the cmake sources for “.vfproj” leads to the documentation at Help/variable/CMAKE_MAKE_PROGRAM.rst · v3.22.0 · CMake. Turns out this is simply the public documentation at CMAKE_MAKE_PROGRAM — CMake 3.23.2 Documentation. Finally get to the generators docs at cmake-generators(7) — CMake 3.22.5.

If the Visual Studio generator is not appropriate, then which one is? Since I’m using MSYS2, I wonder if the MSYS generator is better suited to this build task. Come to think of it, I saw some discussion of makefile generators, e.g. in Cannot build using cmake with gfortran on Windows — the Fortran compiler identification is unknown · Issue #328 · fortran-lang/stdlib. Sure enough, the cmake options docs say -G is how you choose the generator:

cmake -G "MinGW Makefiles" -DWITH_ELMERGUI:BOOL=TRUE -DWITH_MPI:BOOL=FALSE -DCMAKE_INSTALL_PREFIX=../install -DCMAKE_Fortran_COMPILER=D:/dev/Software/msys64/mingw64/bin/gfortran.exe -DCMAKE_Fortran_COMPILER_FORCED:BOOL=TRUE -DBLAS_LIBRARIES=D:/dev/Software/msys64/mingw64/lib -DLAPACK_LIBRARIES=D:/dev/Software/msys64/mingw64/lib ../elmerfem

That does not work though (in my developer command prompt)

CMake Error: CMake was unable to find a build program corresponding to "MinGW Makefiles".  CMAKE_MAKE_PROGRAM is not set.  You probably need to select a different build tool.
CMake Error: CMake was unable to find a build program corresponding to "MinGW Makefiles".  CMAKE_MAKE_PROGRAM is not set.  You probably need to select a different build tool.
CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
-- Configuring incomplete, errors occurred!

Looks like I need to try this process in MSYS2.

Custom Generator in MSYS

Running which cmake in MSYS did not find cmake so here’s the version I installed.

$ pacman -Ss cmake
...
mingw64/mingw-w64-x86_64-cmake 3.23.2-1
    A cross-platform open-source make system (mingw-w64)
...
$ pacman -S mingw64/mingw-w64-x86_64-cmake

This doesn’t result in being able to run cmake.exe (even though it exists on disk in D:\dev\Software\msys64\mingw64\bin). Time to hit the docs again: msys2 cmake – Search (bing.com) -> Using CMake in MSYS2 – MSYS2. No red flags there… How about a search for the exact error message: msys bash: cmake: command not found – Search (bing.com) -> c++ – CMake is not found when running through make – Stack Overflow. Aha! The answer there about launching MSYS2 using mingw32.exe leads me to inquire about how I’m launching MSYS2. Turns out I’m launching using the last shortcut below (which launches “D:\dev\Software\msys64\msys2_shell.cmd -msys“) instead of MinGW x64.lnk (which launches “D:\dev\Software\msys64\msys2_shell.cmd -mingw64“). Sure enough, which cmake now shows /mingw64/bin/cmake.

 Directory of C:\Users\USERNAME\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\MSYS2 64bit

MSYS2 MinGW Clang x64.lnk
MSYS2 MinGW UCRT x64.lnk
MSYS2 MinGW x64.lnk
MSYS2 MinGW x86.lnk
MSYS2 MSYS.lnk

Custom Generator in MinGW

Retrying the command line now makes progress! Notice the Fortran compiler is successfully detected (and the GNU C++ compiler is also selected).

$ cmake -G "MinGW Makefiles" -DWITH_ELMERGUI:BOOL=TRUE -DWITH_MPI:BOOL=FALSE -DCMAKE_INSTALL_PREFIX=../install -DCMAKE_Fortran_COMPILER=D:/dev/Software/msys64/mingw64/bin/gfortran.exe -DCMAKE_Fortran_COMPILER_FORCED:BOOL=TRUE -DBLAS_LIBRARIES=D:/dev/Software/msys64/mingw64/lib -DLAPACK_LIBRARIES=D:/dev/Software/msys64/mingw64/lib ../elmerfem
-- The Fortran compiler identification is GNU 12.1.0
-- The C compiler identification is GNU 12.1.0
-- The CXX compiler identification is GNU 12.1.0
...

The build fails but things are very promising now. The error is because Qt is missing:

--   Building ElmerGUI
-- ------------------------------------------------
CMake Deprecation Warning at ElmerGUI/CMakeLists.txt:1 (CMAKE_MINIMUM_REQUIRED):
  Compatibility with CMake < 2.8.12 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


CMake Warning at ElmerGUI/CMakeLists.txt:19 (find_package):
  By not providing "FindQt5.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "Qt5", but
  CMake did not find one.

  Could not find a package configuration file provided by "Qt5" with any of
  the following names:

    Qt5Config.cmake
    qt5-config.cmake

  Add the installation prefix of "Qt5" to CMAKE_PREFIX_PATH or set "Qt5_DIR"
  to a directory containing one of the above files.  If "Qt5" provides a
  separate development package or SDK, be sure it has been installed.


-- ------------------------------------------------
CMake Error at D:/dev/Software/msys64/mingw64/share/cmake/Modules/FindQt4.cmake:1314 (message):
  Found unsuitable Qt version "" from NOTFOUND, this code requires Qt 4.x
Call Stack (most recent call first):
  ElmerGUI/CMakeLists.txt:42 (FIND_PACKAGE)

Installing Qt5 does not address the build failure. The new error message:

--   Building ElmerGUI
-- ------------------------------------------------
CMake Deprecation Warning at ElmerGUI/CMakeLists.txt:1 (CMAKE_MINIMUM_REQUIRED):
  Compatibility with CMake < 2.8.12 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


-- ------------------------------------------------
-- Qt5 Windows packaging
--   [ElmerGUI] Qt5:               1
--   [ElmerGUI] Qt5 Libraries: Qt5::OpenGL Qt5::Xml Qt5::Script Qt5::Gui Qt5::Core
-- ------------------------------------------------
CMake Warning (dev) at D:/dev/Software/msys64/mingw64/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:438 (message):
  The package name passed to `find_package_handle_standard_args` (OpenGL)
  does not match the name of the calling package (Qwt).  This can lead to
  problems in calling code that expects `find_package` result variables
  (e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
  D:/dev/Software/msys64/mingw64/share/cmake/Modules/FindOpenGL.cmake:443 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  ElmerGUI/cmake/Modules/FindQwt.cmake:10 (INCLUDE)
  ElmerGUI/CMakeLists.txt:61 (FIND_PACKAGE)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Found OpenGL: opengl32
CMake Warning (dev) at D:/dev/Software/msys64/mingw64/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:438 (message):
  The package name passed to `find_package_handle_standard_args` (Qt3) does
  not match the name of the calling package (Qwt).  This can lead to problems
  in calling code that expects `find_package` result variables (e.g.,
  `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
  D:/dev/Software/msys64/mingw64/share/cmake/Modules/FindQt3.cmake:213 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  D:/dev/Software/msys64/mingw64/share/cmake/Modules/FindQt.cmake:160 (include)
  ElmerGUI/cmake/Modules/FindQwt.cmake:11 (INCLUDE)
  ElmerGUI/CMakeLists.txt:61 (FIND_PACKAGE)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Could NOT find Qt3 (missing: QT_QT_LIBRARY QT_INCLUDE_DIR)
CMake was unable to find desired Qt version: 3. Set advanced values QT_QMAKE_EXECUTABLE and QT3_QGLOBAL_H_FILE.
--   [ElmerGUI] Qwt:             FALSE
--   [ElmerGUI] QWT_LIBRARY:     QWT_LIBRARY-NOTFOUND
--   [ElmerGUI] QWT_INCLUDE_DIR: QWT_INCLUDE_DIR-NOTFOUND
-- ------------------------------------------------
CMake Warning (dev) at D:/dev/Software/msys64/mingw64/lib/cmake/Qt5Core/Qt5CoreMacros.cmake:44 (message):
  qt5_use_modules is not part of the official API, and might be removed in Qt
  6.
Call Stack (most recent call first):
  D:/dev/Software/msys64/mingw64/lib/cmake/Qt5Core/Qt5CoreMacros.cmake:431 (_qt5_warn_deprecated)
  ElmerGUI/Application/CMakeLists.txt:216 (QT5_USE_MODULES)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- ------------------------------------------------
--   BLAS library:   D:/dev/Software/msys64/mingw64/lib
--   LAPACK library: D:/dev/Software/msys64/mingw64/lib
-- ------------------------------------------------
--   Fortran compiler:        D:/dev/Software/msys64/mingw64/bin/gfortran.exe
--   Fortran flags:            -fallow-argument-mismatch -O2 -g -DNDEBUG
-- ------------------------------------------------
--   C compiler:              D:/dev/Software/msys64/mingw64/bin/cc.exe
--   C flags:                  -O2 -g -DNDEBUG
-- ------------------------------------------------
--   CXX compiler:            D:/dev/Software/msys64/mingw64/bin/c++.exe
--   CXX flags:                -O2 -g -DNDEBUG
-- ------------------------------------------------
-- ------------------------------------------------
--   Package filename: elmerfem-9.0--20220612_Windows-AMD64
--   Patch version: 9.0-
CMake Error at cpack/ElmerCPack.cmake:99 (INSTALL):
  INSTALL FILES given directory "D:/dev/Software/msys64/mingw64/lib" to
  install.
Call Stack (most recent call first):
  CMakeLists.txt:660 (INCLUDE)


-- Configuring incomplete, errors occurred!
See also "D:/dev/repos/fem/build/CMakeFiles/CMakeOutput.log".
See also "D:/dev/repos/fem/build/CMakeFiles/CMakeError.log".

Does this need Qt3? The ElmerGUI documentation says Qt4 (4.8 or higher). FindQt.cmake:160 (in bold above) appears to indicate that only Qt versions 3 and 4 are supported in MinGW. The mix of warnings and “could not find” makes it hard to know exactly what is wrong. The last error, for example, appears to be about the installation files directory. So is there anything wrong with Qt? I’ll assume not.

The cmake docs on installing files doesn’t point to anything peculiar in this scenario but this is a hint that my LAPACK_LIBRARIES variable is most likely wrong. Let’s drop it altogether:

# Clean up old make files
# rm -fr *

cmake -G "MinGW Makefiles" -DWITH_ELMERGUI:BOOL=TRUE -DWITH_MPI:BOOL=FALSE -DCMAKE_INSTALL_PREFIX=../install -DCMAKE_Fortran_COMPILER=D:/dev/Software/msys64/mingw64/bin/gfortran.exe -DCMAKE_Fortran_COMPILER_FORCED:BOOL=TRUE -DBLAS_LIBRARIES=D:/dev/Software/msys64/mingw64/lib ../elmerfem

The build still fails but right before the error, notice the LAPACK library now has a DLL instead of a directory (below)!

-- ------------------------------------------------
--   BLAS library:   D:/dev/Software/msys64/mingw64/lib
--   LAPACK library: D:/dev/Software/msys64/mingw64/lib/libopenblas.dll.a;D:/dev/Software/msys64/mingw64/lib
-- ------------------------------------------------
--   Fortran compiler:        D:/dev/Software/msys64/mingw64/bin/gfortran.exe
--   Fortran flags:            -fallow-argument-mismatch -O2 -g -DNDEBUG
-- ------------------------------------------------
--   C compiler:              D:/dev/Software/msys64/mingw64/bin/cc.exe
--   C flags:                  -O2 -g -DNDEBUG
-- ------------------------------------------------
--   CXX compiler:            D:/dev/Software/msys64/mingw64/bin/c++.exe
--   CXX flags:                -O2 -g -DNDEBUG
-- ------------------------------------------------
-- ------------------------------------------------
--   Package filename: elmerfem-9.0--20220612_Windows-AMD64
--   Patch version: 9.0-
CMake Error at cpack/ElmerCPack.cmake:99 (INSTALL):
  INSTALL FILES given directory "D:/dev/Software/msys64/mingw64/lib" to
  install.

So now it makes sense to drop the BLAS_LIBRARIES definition as well!

# Clean up old make files
# rm -fr *

cmake -G "MinGW Makefiles" -DWITH_ELMERGUI:BOOL=TRUE -DWITH_MPI:BOOL=FALSE -DCMAKE_INSTALL_PREFIX=../install -DCMAKE_Fortran_COMPILER=D:/dev/Software/msys64/mingw64/bin/gfortran.exe -DCMAKE_Fortran_COMPILER_FORCED:BOOL=TRUE ../elmerfem

This build step now succeeds as indicated by the selection of libopenblas.dll.a as the BLAS and LAPACK library.

-- ------------------------------------------------
--   BLAS library:   D:/dev/Software/msys64/mingw64/lib/libopenblas.dll.a
--   LAPACK library: D:/dev/Software/msys64/mingw64/lib/libopenblas.dll.a
-- ------------------------------------------------
--   Fortran compiler:        D:/dev/Software/msys64/mingw64/bin/gfortran.exe
--   Fortran flags:            -fallow-argument-mismatch -O2 -g -DNDEBUG
-- ------------------------------------------------
--   C compiler:              D:/dev/Software/msys64/mingw64/bin/cc.exe
--   C flags:                  -O2 -g -DNDEBUG
-- ------------------------------------------------
--   CXX compiler:            D:/dev/Software/msys64/mingw64/bin/c++.exe
--   CXX flags:                -O2 -g -DNDEBUG
-- ------------------------------------------------
-- ------------------------------------------------
--   Package filename: elmerfem-9.0--20220612_Windows-AMD64
--   Patch version: 9.0-
-- Configuring done
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
QWT_INCLUDE_DIR (ADVANCED)
   used as include directory in directory D:/dev/repos/fem/elmerfem/ElmerGUI/Application
   ...
   used as include directory in directory D:/dev/repos/fem/elmerfem/ElmerGUI/Application
QWT_LIBRARY (ADVANCED)
    linked by target "ElmerGUI" in directory D:/dev/repos/fem/elmerfem/ElmerGUI/Application
...

Looks like I now need to define QWT_INCLUDE_DIR and QWT_LIBRARY. Hmm, I don’t think I even installed QWT.

$ pacman -S mingw64/mingw-w64-x86_64-qwt-qt5
resolving dependencies...
looking for conflicting packages...

Packages (1) mingw-w64-x86_64-qwt-qt5-6.2.0-5

Total Download Size:    29.17 MiB
Total Installed Size:  175.53 MiB

:: Proceed with installation? [Y/n] y
:: Retrieving packages...
 mingw-w64-x86_64-qwt-qt5-6.2.0-5-any                                                                                  29.2 MiB  1136 KiB/s 00:26 [###...###] 100%
(1/1) checking keys in keyring                                                                                                                    [###...###] 100%
(1/1) checking package integrity                                                                                                                  [###...###] 100%
(1/1) loading package files                                                                                                                       [###...###] 100%
(1/1) checking for file conflicts                                                                                                                 [###...###] 100%
(1/1) checking available disk space                                                                                                               [###...###] 100%
:: Processing package changes...
(1/1) installing mingw-w64-x86_64-qwt-qt5                                                                                                         [#########################################################################################] 100%
Optional dependencies for mingw-w64-x86_64-qwt-qt5
    mingw-w64-x86_64-qt5-tools [installed]

Now that QWT is installed, we can set the include directory as follows:

cmake -G "MinGW Makefiles" -DWITH_ELMERGUI:BOOL=TRUE -DWITH_MPI:BOOL=FALSE -DCMAKE_INSTALL_PREFIX=../install -DCMAKE_Fortran_COMPILER=D:/dev/Software/msys64/mingw64/bin/gfortran.exe -DCMAKE_Fortran_COMPILER_FORCED:BOOL=TRUE -DQWT_INCLUDE_DIR=D:/dev/Software/msys64/mingw64/include/qwt-qt5/ ../elmerfem

CMake finally succeeds! The output ends with these lines:

-- Generating done
-- Build files have been written to: D:/dev/repos/fem/build

The generated Makefile has targets such as ElmerGUI, elmersolver, AdvectionDiffusion, FluxSolver, etc. The strange thing is that it has a line that sets SHELL = cmd.exe and so a Windows command prompt is launched when you run make.

#==================================================================
# Target rules for targets named ElmerGUI

# Build rule for target.
ElmerGUI: cmake_check_build_system
	$(MAKE) $(MAKESILENT) -f CMakeFiles\Makefile2 ElmerGUI
.PHONY : ElmerGUI

# fast build rule for target.
ElmerGUI/fast:
	$(MAKE) $(MAKESILENT) -f ElmerGUI\Application\CMakeFiles\ElmerGUI.dir\build.make ElmerGUI/Application/CMakeFiles/ElmerGUI.dir/build
.PHONY : ElmerGUI/fast

Some digging around via mingw cmake shell at DuckDuckGo and I’m reading that makefiles from the MinGW Makefiles generator are for use with mingw32-make under a Windows command prompt. Looks like I need the MSYS Makefiles generator.

cmake -G "MSYS Makefiles" -DWITH_ELMERGUI:BOOL=TRUE -DWITH_MPI:BOOL=FALSE -DCMAKE_INSTALL_PREFIX=../install -DCMAKE_Fortran_COMPILER=D:/dev/Software/msys64/mingw64/bin/gfortran.exe -DCMAKE_Fortran_COMPILER_FORCED:BOOL=TRUE -DQWT_INCLUDE_DIR=D:/dev/Software/msys64/mingw64/include/qwt-qt5/ ../elmerfem

Now we see the expected SHELL = /bin/sh and running make actually causes code to start building! What a journey! I will write another post with simplified instructions for how to build Elmer (on Windows).

$ make
[  0%] Building C object matc/src/CMakeFiles/matc.dir/c3d.c.obj
[  0%] Building C object matc/src/CMakeFiles/matc.dir/clip.c.obj
[  0%] Building C object matc/src/CMakeFiles/matc.dir/dri_ps.c.obj
[  0%] Building C object matc/src/CMakeFiles/matc.dir/eig.c.obj
...