We need to clone 2 repos to execute the crystal growth experiment. The opencgs repo is a dependency of the test-cz-induction repo.
cd ~/repos/fem/elmer/research
git clone https://github.com/nemocrys/opencgs
git clone https://github.com/nemocrys/test-cz-induction
Next, install the opencgs module using pip then set up the crystal growth simulation mesh.
cd opencgs
pip install -e .
This installation output contains an error about an incompatible numpy version but ends with a message about all the components being successfully installed.
Requirement already satisfied: kiwisolver>=1.0.1 in /home/saint/.local/lib/python3.8/site-packages (from matplotlib->pyelmer->opencgs==0.3.1) (1.4.4)
Requirement already satisfied: fonttools>=4.22.0 in /home/saint/.local/lib/python3.8/site-packages (from matplotlib->pyelmer->opencgs==0.3.1) (4.34.4)
ERROR: pandas 1.4.3 has requirement numpy>=1.18.5; platform_machine != "aarch64" and platform_machine != "arm64" and python_version < "3.10", but you'll have numpy 1.17.4 which is incompatible.
...
Successfully installed commonmark-0.9.1 meshio-5.3.4 opencgs pandas-1.4.3 pygments-2.12.0 python-dateutil-2.8.2 pytz-2022.1 rich-12.5.1 typing-extensions-4.3.0
I ignore the error and forge ahead with running the crystal simulation setup:
cd ../test-cz-induction
python3 setup.py
This fails with an error related to numpy.
python3 setup.py
Traceback (most recent call last):
File "setup.py", line 19, in <module>
import opencgs.control as ctrl
File "/home/saint/repos/fem/elmer/research/opencgs/opencgs/__init__.py", line 5, in <module>
import opencgs.post
File "/home/saint/repos/fem/elmer/research/opencgs/opencgs/post.py", line 2, in <module>
import meshio
File "/home/saint/.local/lib/python3.8/site-packages/meshio/__init__.py", line 1, in <module>
from . import (
File "/home/saint/.local/lib/python3.8/site-packages/meshio/_cli/__init__.py", line 1, in <module>
from ._main import main
File "/home/saint/.local/lib/python3.8/site-packages/meshio/_cli/_main.py", line 5, in <module>
from . import _ascii, _binary, _compress, _convert, _decompress, _info
File "/home/saint/.local/lib/python3.8/site-packages/meshio/_cli/_ascii.py", line 4, in <module>
from .. import ansys, flac3d, gmsh, mdpa, ply, stl, vtk, vtu, xdmf
File "/home/saint/.local/lib/python3.8/site-packages/meshio/ansys/__init__.py", line 1, in <module>
from ._ansys import read, write
File "/home/saint/.local/lib/python3.8/site-packages/meshio/ansys/_ansys.py", line 14, in <module>
from .._helpers import register_format
File "/home/saint/.local/lib/python3.8/site-packages/meshio/_helpers.py", line 7, in <module>
from numpy.typing import ArrayLike
$ pip install numpy --upgrade
Collecting numpy
Downloading numpy-1.23.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.1 MB)
|████████████████████████████████| 17.1 MB 3.6 MB/s
Installing collected packages: numpy
WARNING: The scripts f2py, f2py3 and f2py3.8 are installed in '/home/saint/.local/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed numpy-1.23.1
That’s all that’s needed to get things to run. For example, setting up the mesh now works:
python3 setup.py
The crystal growth simulation can now be executed as well:
python3 run.py
Interestingly, the simulation completes successfully!
crucible
melt
crystal
inductor
seed
insulation
crucible_adapter
axis_bt
...
using material graphite-CZ3R6300 from self.materials_dict
using material insulation from self.materials_dict
using material steel-1.4541 from self.materials_dict
using material vacuum from self.materials_dict
Wrote sif-file.
Starting simulation ./simdata/2022-07-31_13-35_ss_test-cz-induction_vacuum ...
[] [] {'CPU-time': 55.87, 'real-time': 57.49}
Finished simulation ./simdata/2022-07-31_13-35_ss_test-cz-induction_vacuum .
Post processing...
evaluating heat fluxes
Finished post processing.
This implies that the segmentation fault I ran into on Windows was specific to the Windows Elmer build.
Visualization
Use ParaView to visualize the results of the simulation.
sudo apt install paraview
There are different variables that can be visualized. I selected temperature for the screenshot below. Other options included joule field, joule heating, potential im, newy, etc.
After completing the mesh generation in Crystal Growth Simulation – Part 1, I decided to investigate how to run the actual crystal growth simulation using Elmer. On Windows, Elmer needs to either be installed into “Program Files“ or be present in the path to avoid this error:
$ python run.py
crucible
melt
crystal
inductor
...
using material steel-1.4541 from self.materials_dict
using material vacuum from self.materials_dict
Wrote sif-file.
Starting simulation ./simdata/2022-07-25_13-17_ss_test-cz-induction_vacuum ...
Traceback (most recent call last):
File "D:\dev\repos\fem\research\test-cz-induction\run.py", line 32, in <module>
sim.execute()
File "d:\dev\repos\fem\research\opencgs\opencgs\sim.py", line 182, in execute
run_elmer_grid(self.sim_dir, "case.msh")
File "C:\Python310\lib\site-packages\pyelmer\execute.py", line 28, in run_elmer_grid
subprocess.run(args, cwd=sim_dir, stdout=f, stderr=f)
File "C:\Python310\lib\subprocess.py", line 501, in run
with Popen(*popenargs, **kwargs) as process:
File "C:\Python310\lib\subprocess.py", line 966, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "C:\Python310\lib\subprocess.py", line 1435, in _execute_child
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2] The system cannot find the file specified
I tried installing a locally built Elmer – see How to Build Elmer on Windows. Unfortunately, the layout created by setup defaulted to “C:\Program Files\Elmer 9.0-1b8e4f7ec” (ending with the hash of the commit I built, instead of “Release”). Worse still, it didn’t launch because of missing DLLs. I need to take a closer look at how NSIS is creating the setup executable, and also figure out how to set up the publisher so that a certificate is displayed with the UAC prompt. I ended up installing the publicly downloadable Elmer then retrying the python script.
using material steel-1.4541 from self.materials_dict
using material vacuum from self.materials_dict
Wrote sif-file.
Starting simulation ./simdata/2022-07-25_17-23_ss_test-cz-induction_vacuum_1 ...
['ERROR:: systemc: Command exit status was 1'] [] {}
Finished simulation ./simdata/2022-07-25_17-23_ss_test-cz-induction_vacuum_1 .
Post processing...
Traceback (most recent call last):
File "D:\dev\repos\fem\research\test-cz-induction\run.py", line 32, in <module>
sim.execute()
File "d:\dev\repos\fem\research\opencgs\opencgs\sim.py", line 189, in execute
self._postprocessing_probes()
File "d:\dev\repos\fem\research\opencgs\opencgs\sim.py", line 139, in _postprocessing_probes
names_data = self._read_names_file(self.sim_dir + "/results/probes.dat.names")
File "d:\dev\repos\fem\research\opencgs\opencgs\sim.py", line 129, in _read_names_file
with open(names_file) as f:
FileNotFoundError: [Errno 2] No such file or directory: './simdata/2022-07-25_17-23_ss_test-cz-induction_vacuum_1/02_simulation/results/probes.dat.names'
A dialog popped up asking whether I wanted to grant some elmer process access to the domain network or private, etc, and I’m not sure if I took too long to answer it but the script continued with the above failure. This looks like a bug in opencgs – why continue post processing when the simulation has failed – I filed this issue to let the author know: Do not start post-processing when simulation fails · Issue #2 · nemocrys/opencgs (github.com).
I had no idea why the simulation is failing so I used Process Explorer to see which command line was being used to invoke ElmerGrid and ElmerSolver. I was able to then manually invoke ElmerSolver and notice a segmentation fault!
ELMER SOLVER (v 9.0) STARTED AT: 2022/07/25 17:23:13
ParCommInit: Initialize #PEs: 1
MAIN:
MAIN: =============================================================
MAIN: ElmerSolver finite element software, Welcome!
...
MAIN: Reading Model: case.sif
Caught LUA error:[string "loadfile("C:/Program Files (x86)/Elmer/shar..."]:1: attempt to call a nil value
Caught LUA error:[string "loadstring(readsif("case.sif"))()"]:1: attempt to call global 'readsif' (a nil value)
LoadInputFile: Scanning input file: case.sif
LoadInputFile: Scanning only size info
...
OptimizeBandwidth: Half bandwidth after optimization: 1447
OptimizeBandwidth: ---------------------------------------------------------
'ViewFactors' is not recognized as an internal or external command,
operable program or batch file.
ERROR:: systemc: Command exit status was 1
RadiationFactors: Message
RadiationFactors: All done time (s) 4.9000E-02
...
60 0.2041E-08
61 0.3710E-09
ComputeChange: NS (ITER=11) (NRM,RELC): ( 0.17032920E-04 0.41334346E-05 ) :: statmagsolver
StatMagSolve: Convergence after 11 iterations
StatMagSolve: Joule Heating (W): 4.1375E+01
StatMagSolve: All done, exiting
StatMagSolve: ------------------------------------------------
ComputeChange: SS (ITER=1) (NRM,RELC): ( 0.17032920E-04 2.0000000 ) :: statmagsolver
HeatSolve: Found Control Point at distance: 0.0000000000000000
HeatSolve: Control Point Index: 0
HeatSolve: Using Steady-state Heater Control
HeatSolve:
HeatSolve:
HeatSolve: -------------------------------------
HeatSolve: TEMPERATURE ITERATION 1
HeatSolve: -------------------------------------
HeatSolve:
HeatSolve: Starting Assembly...
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
#0 0xffffffff
#1 0xffffffff
#2 0xffffffff
...
#19 0xffffffff
#20 0xffffffff
Looking at the simulation folders such as “simdata\2022-07-25_22-52_ss_test-cz-induction_vacuum_1\02_simulation” revealed that there are ElmerGrid and ElmerSolver log files present with this info, so I didn’t even need Process Explorer! At this point, I have to save the SIGSEGV investigation for another day.
Ever since I read chapter 2 of Fabrication Engineering at the Micro- and Nanoscale, I have been interested in simulating crystal growth, primarily because of the multi-discplinary nature of this problem: finite element modeling software, modeling using high performance computing, the physics of the problem (governing laws and equations), and visualization techniques. I had run across Elmer when looking up crystal growth examples a while back. About two weeks ago, I was watching this Elmer demo in which they used gmsh to show the geometry for the simulation.
I wanted to learn more about Gmsh and how it is used, so I took a detour into this Gmsh introductory video. It was an excellent tour of Gmsh and its features.
Thankfully, the code in this paper is open source. I already had python installed so the first thing I tried was executing run.py in Git bash
$ git clone https://github.com/nemocrys/test-cz-induction
$ cd test-cz-induction
$ python run.py
Traceback (most recent call last):
File "D:\dev\repos\fem\research\test-cz-induction\run.py", line 5, in <module>
import opencgs.control as ctrl
ModuleNotFoundError: No module named 'opencgs'
Without using my brain, I decided to install pyelmer since I knew it was a dependency (from the talk).
$ pip install pyelmer
Collecting pyelmer
Downloading pyelmer-1.0.0-py3-none-any.whl (27 kB)
Collecting gmsh
Downloading gmsh-4.10.5-py2.py3-none-win_amd64.whl (38.4 MB)
|████████████████████████████████| 38.4 MB 285 kB/s
Collecting pyyaml
Downloading PyYAML-6.0-cp310-cp310-win_amd64.whl (151 kB)
|████████████████████████████████| 151 kB ...
Collecting matplotlib
Downloading matplotlib-3.5.2-cp310-cp310-win_amd64.whl (7.2 MB)
|████████████████████████████████| 7.2 MB 6.4 MB/s
Collecting pillow>=6.2.0
Downloading Pillow-9.2.0-cp310-cp310-win_amd64.whl (3.3 MB)
|████████████████████████████████| 3.3 MB ...
Collecting cycler>=0.10
Downloading cycler-0.11.0-py3-none-any.whl (6.4 kB)
Collecting numpy>=1.17
Downloading numpy-1.23.1-cp310-cp310-win_amd64.whl (14.6 MB)
|████████████████████████████████| 14.6 MB 6.4 MB/s
Collecting packaging>=20.0
Downloading packaging-21.3-py3-none-any.whl (40 kB)
|████████████████████████████████| 40 kB 2.5 MB/s
Collecting python-dateutil>=2.7
Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
|████████████████████████████████| 247 kB ...
Collecting kiwisolver>=1.0.1
Downloading kiwisolver-1.4.4-cp310-cp310-win_amd64.whl (55 kB)
|████████████████████████████████| 55 kB 1.6 MB/s
Collecting pyparsing>=2.2.1
Downloading pyparsing-3.0.9-py3-none-any.whl (98 kB)
|████████████████████████████████| 98 kB 6.8 MB/s
Collecting fonttools>=4.22.0
Downloading fonttools-4.34.4-py3-none-any.whl (944 kB)
|████████████████████████████████| 944 kB 6.4 MB/s
Collecting six>=1.5
Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Installing collected packages: six, pyparsing, python-dateutil, pillow, packaging, numpy, kiwisolver, fonttools, cycler, pyyaml, matplotlib, gmsh, pyelmer
WARNING: Failed to write executable - trying to use .deleteme logic
ERROR: Could not install packages due to an OSError: [WinError 2] The system cannot find the file specified: 'C:\\Python310\\Scripts\\f2py.exe' -> 'C:\\Python310\\Scripts\\f2py.exe.deleteme'
WARNING: You are using pip version 21.2.4; however, version 22.2 is available.
You should consider upgrading via the 'C:\Python310\python.exe -m pip install --upgrade pip' command.
Looks like I need to run this as an administrator.
C:\dev>pip install pyelmer
Collecting pyelmer
Using cached pyelmer-1.0.0-py3-none-any.whl (27 kB)
Collecting pyyaml
Using cached PyYAML-6.0-cp310-cp310-win_amd64.whl (151 kB)
Collecting gmsh
Using cached gmsh-4.10.5-py2.py3-none-win_amd64.whl (38.4 MB)
Collecting matplotlib
Using cached matplotlib-3.5.2-cp310-cp310-win_amd64.whl (7.2 MB)
Requirement already satisfied: python-dateutil>=2.7 in c:\python310\lib\site-packages (from matplotlib->pyelmer) (2.8.2)
Requirement already satisfied: packaging>=20.0 in c:\python310\lib\site-packages (from matplotlib->pyelmer) (21.3)
Requirement already satisfied: fonttools>=4.22.0 in c:\python310\lib\site-packages (from matplotlib->pyelmer) (4.34.4)
Requirement already satisfied: kiwisolver>=1.0.1 in c:\python310\lib\site-packages (from matplotlib->pyelmer) (1.4.4)
Requirement already satisfied: pillow>=6.2.0 in c:\python310\lib\site-packages (from matplotlib->pyelmer) (9.2.0)
Requirement already satisfied: pyparsing>=2.2.1 in c:\python310\lib\site-packages (from matplotlib->pyelmer) (3.0.9)
Requirement already satisfied: numpy>=1.17 in c:\python310\lib\site-packages (from matplotlib->pyelmer) (1.23.1)
Collecting cycler>=0.10
Using cached cycler-0.11.0-py3-none-any.whl (6.4 kB)
Requirement already satisfied: six>=1.5 in c:\python310\lib\site-packages (from python-dateutil>=2.7->matplotlib->pyelmer) (1.16.0)
Installing collected packages: cycler, pyyaml, matplotlib, gmsh, pyelmer
Successfully installed cycler-0.11.0 gmsh-4.10.5 matplotlib-3.5.2 pyelmer-1.0.0 pyyaml-6.0
WARNING: You are using pip version 21.2.4; however, version 22.2 is available.
You should consider upgrading via the 'C:\Python310\python.exe -m pip install --upgrade pip' command.
Installing pyelmer is not sufficient to avoid the “No module named ‘opencgs’” error. The solution is to clone the opencgs repo and use pip to install (as administrator) from that repo.
git clone https://github.com/nemocrys/opencgs
cd opencgs
D:\...\research\opencgs>pip install -e .
Obtaining file:///D:/dev/repos/fem/research/opencgs
Collecting meshio
Using cached meshio-5.3.4-py3-none-any.whl (167 kB)
Collecting pandas
Using cached pandas-1.4.3-cp310-cp310-win_amd64.whl (10.5 MB)
Requirement already satisfied: pyyaml in c:\python310\lib\site-packages (from opencgs==0.3.1) (6.0)
Requirement already satisfied: pyelmer in c:\python310\lib\site-packages (from opencgs==0.3.1) (1.0.0)
Collecting rich
Using cached rich-12.5.1-py3-none-any.whl (235 kB)
Requirement already satisfied: numpy in c:\python310\lib\site-packages (from meshio->opencgs==0.3.1) (1.23.1)
Requirement already satisfied: python-dateutil>=2.8.1 in c:\python310\lib\site-packages (from pandas->opencgs==0.3.1) (2.8.2)
Collecting pytz>=2020.1
Using cached pytz-2022.1-py2.py3-none-any.whl (503 kB)
Requirement already satisfied: six>=1.5 in c:\python310\lib\site-packages (from python-dateutil>=2.8.1->pandas->opencgs==0.3.1) (1.16.0)
Requirement already satisfied: matplotlib in c:\python310\lib\site-packages (from pyelmer->opencgs==0.3.1) (3.5.2)
Requirement already satisfied: gmsh in c:\python310\lib\site-packages (from pyelmer->opencgs==0.3.1) (4.10.5)
Requirement already satisfied: kiwisolver>=1.0.1 in c:\python310\lib\site-packages (from matplotlib->pyelmer->opencgs==0.3.1) (1.4.4)
Requirement already satisfied: pyparsing>=2.2.1 in c:\python310\lib\site-packages (from matplotlib->pyelmer->opencgs==0.3.1) (3.0.9)
Requirement already satisfied: cycler>=0.10 in c:\python310\lib\site-packages (from matplotlib->pyelmer->opencgs==0.3.1) (0.11.0)
Requirement already satisfied: pillow>=6.2.0 in c:\python310\lib\site-packages (from matplotlib->pyelmer->opencgs==0.3.1) (9.2.0)
Requirement already satisfied: fonttools>=4.22.0 in c:\python310\lib\site-packages (from matplotlib->pyelmer->opencgs==0.3.1) (4.34.4)
Requirement already satisfied: packaging>=20.0 in c:\python310\lib\site-packages (from matplotlib->pyelmer->opencgs==0.3.1) (21.3)
Collecting commonmark<0.10.0,>=0.9.0
Using cached commonmark-0.9.1-py2.py3-none-any.whl (51 kB)
Requirement already satisfied: pygments<3.0.0,>=2.6.0 in c:\python310\lib\site-packages (from rich->meshio->opencgs==0.3.1) (2.12.0)
Installing collected packages: commonmark, rich, pytz, pandas, meshio, opencgs
Running setup.py develop for opencgs
Successfully installed commonmark-0.9.1 meshio-5.3.4 opencgs-0.3.1 pandas-1.4.3 pytz-2022.1 rich-12.5.1
WARNING: You are using pip version 21.2.4; however, version 22.2 is available.
You should consider upgrading via the 'C:\Python310\python.exe -m pip install --upgrade pip' command.
Trying to run gives an error about a missing module named objectgmsh. My assumption was the installing pyelmer was supposed to bring in all such modules.
D:\dev\repos\fem\research\test-cz-induction>python run.py
Traceback (most recent call last):
File "D:\dev\repos\fem\research\test-cz-induction\run.py", line 5, in <module>
import opencgs.control as ctrl
File "d:\dev\repos\fem\research\opencgs\opencgs\__init__.py", line 3, in <module>
import opencgs.geo
File "d:\dev\repos\fem\research\opencgs\opencgs\geo.py", line 1, in <module>
from objectgmsh import *
ModuleNotFoundError: No module named 'objectgmsh'
I notice while reviewing the paper yet again that it mentions the setup being in a Dockerfile. Sure enough, objectgmsh is a separate package that needs to be installed using pip. I upgrade pip for good measure since those warnings are getting annoying.
