Last week I bought a new MacBook Pro with the Apple M1 Chip and 16GB of RAM. These are the steps I used to set it up for building the OpenJDK codebase.
Set your Mac’s name (it bothers me when the terminal has some random host name). You might need to restart your terminal for this change to take effect.
Install Development Tools
Should any of the commands below fail, see the troubleshooting section at the end for possible workarounds.
Install homebrew by running the recommended command (and see the troubleshooting section if there are any git errors):
Select the Xcode command line tools: launch Xcode then go to Preferences > Locations. Select Xcode 13.1 in the Command Line Tools dropdown as shown below.
Running bash configure in the JDK folder should display any missing dependencies. Any errors from bash configure will need to be resolved before running make images.
cd ~/repos/jdk
bash configure
make images
To browse through the contents of the build folder in finder:
open ~/repos/jdk/build/
To try out your new build, switch to the bin folder and check the Java version:
cd ~/repos/jdk/build/macosx-aarch64-server-release/jdk/bin
./java -version
Here is the output I get:
saint@Saints-MBP-2021 bin % ./java -version
openjdk version "18-internal" 2022-03-22
OpenJDK Runtime Environment (build 18-internal+0-adhoc.saint.jdk)
OpenJDK 64-Bit Server VM (build 18-internal+0-adhoc.saint.jdk, mixed mode)
saint@Saints-MBP-2021 bin %
Troubleshooting
Homebrew Installation Failure
Installing homebrew appeared to be successful (last message output below) but there was a git error in the output!
==> Downloading and installing Homebrew..
remote: Enumerating objects: 345, done.
remote: Counting objects: 100% (297/297), done.
remote: Compressing objects: 100% (107/107), done.
remote: Total 345 (delta 227), reused 238 (delta 184), pack-reused 48
Receiving objects: 100% (345/345), 171.73 KiB | 971.00 KiB/s, done.
Resolving deltas: 100% (227/227),
completed with 54 local objects.
From https://github.com/Homebrew/brew
* [new branch]
dependabot/bundler/Librarv/Homebrew/sorbet-0.5.9396
-> origin/dependabot/bundler/Librarv/Homebrew/sorbet-0.5.9396
778de69b0..be908f679 master -> origin/master
* [new tag] 3.3.6 -> 3.3.6
HEAD is now at be908f679 Merge pull request #12502 from carlocab/bug-template
error: Not a valid ref: refs/remotes/origin/master
fatal: ambiguous argument
'refs/remotes/origin/master': unknown revision or path not in the working tree.
Use
"_-' to separate paths from revisions, like this:
'git <command> [<revision>...] - I<files...11
fatal: Could not resolve HEAD to a revision
Warning: /opt/homebrew/bin is not in your PATH.
Instructions on how to configure vour shell for Homebrew
can be found in the 'Next steps' section below.
==> Installation successful!
Here is the relevant error, which I was able to copy/paste (with some typos) from the PNG!!!
error: Not a valid ref: refs/remotes/origin/master
fatal: ambiguous argument
'refs/remotes/origin/master': unknown revision or path not in the working tree.
I ran into errors of the form No available formula with the name "autoconf" when attempting to install autoconf. However, this happen with the unresolved brew installation git issue described above. Once that was resolved, https://stackoverflow.com/questions/11552171/cant-install-software-using-brew-on-my-mac helpfully pointed out that autoconf is part of the command line tools package (hence step 4 in the instructions above).
[saint@Saints-MBP-2021 jk % brew install autoconf
fatal: Could not resolve HEAD to a revision
Running 'brew update --preinstall'
==> Homebrew is run entirelv by unpaid volunteers. Please consider donating:
https://github.com/Homebrew/brew#donations
==› Auto-updated Homebrew!
Updated 1 tap (homebrew/cask).
==> Updated Casks
dated 1 cask.
Warning: No available formula with the name
"autoconf"
==› Searching for similarlv named formulae.
Error: No similarly named formulae found.
==> Searching for a previously deleted formula (in the last month).
Error: No previously deleted formula found.
==> Searching taps on GitHub.
Error: No formulae found in taps.
No Such File or Directory @ dir_chdir
Setting up a build environment on my Intel MacBook Pro led to errors like this:
==> Installing autoconf dependency: m4
Error: No such file or directory @ dir_chdir - /usr/local/Cellar
One of my bash configure runs failed with this error:
checking for sdk name..
configure: error: No xcodebuild tool and no system framework headers found, use --with-sysroot or --with-sdk-name to provide a path to a valid SDK
/Users/saint/repos/idk/build/.configure-support/generated-configure.sh: line 84: 5: Bad file descriptor
configure exiting with result code 1
I’m using a Windows 10 physical machine for my OpenJDK 17 development. Unfortunately, I ran into some issues getting the environment set up to build the JDK on Windows. To work around this, I created a Linux virtual machine. Although the instructions for building on Linux are on the OpenJDK site, I would like to have all the instructions in one spot, hence this post.
# Valid values are only '18.04' and '20.04'
# For other versions of Ubuntu, please use the tar.gz package
ubuntu_release=`lsb_release -rs`
cd ~/Downloads/
wget https://packages.microsoft.com/config/ubuntu/${ubuntu_release}/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
sudo dpkg -i packages-microsoft-prod.deb
sudo apt-get install apt-transport-https
Verify that everything is working by running “java -version”
Clone and Build the JDK
Clone the JDK. Note that cloning a fork might be much slower than cloning the upstream Github repo! I was averaging about 60KiB/s on my rork whereas cloning the upstream OpenJDK was averaging 6 MiB/s when receiving objects!
mkdir ~/repos
cd ~/repos
git clone https://github.com/openjdk/jdk
The JDK repo can now be configured and built
cd jdk
bash configure
make images
The configure command should display any missing dependencies that it needs and a suggestion for how to install them.
To try out your new build, switch to the bin folder and check the Java version:
cd ~/repos/jdk/build/linux-x86_64-server-release/jdk/bin
./java -version
Last week was my first week in the Java engineering group. It has been about 11 years since I took a compiler course (while in the CS MS program at BYU). A quick review of the history of Java was in order. Turns out I last used Java in 2012 in grad school. That must have been Java SE 7 from 2011 and Java SE 6 before that. Since I have not been in the compiler space since then, I have a steep learning curve ahead. That is the exciting thing about technology though – there is always more to learn!
I am currently a programmer in the developer division at Microsoft so it was helpful going through some of the Java development with Microsoft documentation for a high level overview of all our offerings. Also informative given my long absence from Java-land were the docs on how to Transition from Java 7 to Java 8 and from Java 8 to Java 11. It hadn’t yet dawned on me by the time I read through these that the reason references to 8, 11, and 17 keep coming up is because they are LTS releases.
As a newbie to the Java development world, I started by watching this 2019 OpenJDK Development talk on how to become an OpenJDK contributor. It is a great overview of concepts like project roles (author, committer, reviewer, etc), the contributor agreement, and (perhaps most importantly to me), how to find an issue to work on and build the OpenJDK. The breakdown of commonly used terminology and abbreviations was great to have as well.
For an introduction to the hotspot compiler, I started going through “A Simple Graph-Based Intermediate Representation“. I ended up watching Cliff Click’s talk on The Sea of Nodes and the HotSpot JIT before I got that far along in the paper. It was fascinating seeing details such as the CPU L1/L2 cache size playing into the design! Some of the concepts that I need to review after that talk include:
The sea of nodes talk also revealed to me how little I know about companies in the Java space. I don’t think I had heard of Azul before, for example. In fact, it’s not just companies but also technologies! I was going through some build documentation when I ran into mentions of AdoptOpenJDK and Adoptium, both of which were foreign to me. I was glad though to see my old friend Eclipse doing well.
One of the most enjoyable things about being a programmer is working with very skilled people, especially watching them in action! I always learn a lot! My colleagues David and Mat were kind enough to pull me into their triage and reporting of [JDK-8277299] STACK_OVERFLOW in Java_sun_awt_shell_Win32ShellFolder2_getIconBits – Java Bug System so I could get my feet wet with how things are done in OpenJDK development.
The OpenJDK process is certainly different from the other open source communities I’ve been a part of (.NET and Mozilla Firefox). My manager and I poked around the bug DB to see what compiler starter bugs are out there. I picked bug [JDK-7077093] labelOper::label() should return Label& but since I must start out as an author, issues cannot be assigned to me. Unusual to me but the logic appears sound. Here is the query for C2 starter bugs.
Other highlights of the week were setting up my dev box to build the OpenJDK source code (unsuccessfully), discovering that compiler explorer is a thing (and an open source one at that), learning from my teammates how to investigate a failure of a fairly complex test on MacOS (they were using LLDB). I hope to write follow-up entries on these at some point.
The first programming assignment in the Operating Systems course can be a challenge for students that haven’t written C++ code in a while. While working with the std::queue data structure in C++, it’s easy to make certain types of mistakes (especially if C++ isn’t your native tongue):
Not “using namespace std” when using standard library containers. This can result in some ugly error messages in Visual Studio, e.g. error/warning codes C2143, C4430, and C2238 for the class member array below (is there a better way for students/developers to find out what is happening when they make such a trivial mistake)?
Not understanding the assignment operator semantics on a container like a queue. If we write queue<type> myqueue = array[i]; we get a copy of the queue array[i] (we might have simply wanted a reference/alias). For such a mistake, the code obviously compiles but doesn’t function as intended.
Declaring a fixed-sized data structure to hold all values from a variable-sized container! Runtime errors take care of informing students about this bug (if they’re not lucky enough to have almost empty variable-size containers). The correct declaration of dynamic arrays of templated items is also not usually obvious: T* all_elements = new T[dynamic_integer_size];
For my Computer Aided Geometric Design course, a simple program called CPLOT is used for some of the project work. Its documentation is on the course website. After my initial draft of my project 1 implementation crashed, I was inspired to try out Valgrind on CPLOT (even though I was able to find the bugs using my debugger). All that’s needed to install Valgrind on Ubuntu is sudo apt-get install valgrind. The CPLOT code supplied needed a minor change in order to compile, so I changed
The eg1.dat file I used is the one on page 3 of the CPLOT documentation. Below is the output from valgrind:
==1594== Memcheck, a memory error detector
==1594== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al.
==1594== Using Valgrind-3.6.1 and LibVEX; rerun with -h for copyright info
==1594== Command: ./cplot.out eg1.dat eg1.eps
==1594==
In initps: eg1.eps
File processed successfully!
==1594==
==1594== HEAP SUMMARY:
==1594== in use at exit: 120 bytes in 6 blocks
==1594== total heap usage: 8 allocs, 2 frees, 824 bytes allocated
==1594==
==1594== 120 (8 direct, 112 indirect) bytes in 1 blocks are definitely lost in loss record 3 of 3
==1594== at 0x402641D: operator new(unsigned int) (vg_replace_malloc.c:255)
==1594== by 0x8049224: readCurve() (cplot.cpp:182)
==1594== by 0x8048D89: main (cplot.cpp:127)
==1594==
==1594== LEAK SUMMARY:
==1594== definitely lost: 8 bytes in 1 blocks
==1594== indirectly lost: 112 bytes in 5 blocks
==1594== possibly lost: 0 bytes in 0 blocks
==1594== still reachable: 0 bytes in 0 blocks
==1594== suppressed: 0 bytes in 0 blocks
==1594==
==1594== For counts of detected and suppressed errors, rerun with: -v
==1594== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 17 from 6)
This program can be improved by changing the return statements in the body of the main for(;;) loop into break statements. By so doing, all the cleanup code can be placed after that loop, just before the program exits. This also prevents the duplication of the cleanup code. The code after the for(;;) loop then becomes:
// Deallocate all allocated memory
for (int i=0; i < NCURVES; i++) {
delete curves[i];
}
// Close the file resources
fclose(in);
fclose(ps);
return 0;
==6539== Memcheck, a memory error detector
==6539== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al.
==6539== Using Valgrind-3.6.1 and LibVEX; rerun with -h for copyright info
==6539== Command: ./cplot.out eg1.dat eg1.eps
==6539==
In initps: eg1.eps
File processed successfully!
==6539==
==6539== HEAP SUMMARY:
==6539== in use at exit: 0 bytes in 0 blocks
==6539== total heap usage: 8 allocs, 8 frees, 824 bytes allocated
==6539==
==6539== All heap blocks were freed -- no leaks are possible
==6539==
==6539== For counts of detected and suppressed errors, rerun with: -v
==6539== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 17 from 6)
There was another bug in the program since it could attempt to plot control polygons for uninitialized curves because of a missing else. I pushed the trivial fix to that. The program also crashed if the input file specified did not exist because fscanf would try to use a NULL parameter. The fix is a simple NULL check.
A sample implementation of the dining philosophers problem is provided in the JPF core source code. The solution I gave to this problem was to use the following locking scheme: if a thread has an even id i, then it should lock fork i first and then fork (i + 1) % N. If its id i is odd, then it should lock fork (i + 1) % N first and then lock fork i. Replacing
Part of my Google Summer of Code project this summer involved building a visualization tool to help examine trace files used by JPF Guided Test and a related under-approximation scheduler. The visualization tool is in the Guided Test repository. I was using Eclipse to manage the project. Everything built correctly but the program didn’t work because the icons were not in the build folder as expected. A quick hint from stackoverflow was that the solution is to use Ant.
Double click on the JRE for which you want to attach source code.
Under “JRE System Libraries”, select rt.jar.
Click on the “Source Attachment…” button.
Supply a path to the source zip file, e.g. C:/Program Files/Java/jdk1.6.0_23/src.zip
That zip file was already on my system (since the JDK was installed) but some projects were using the JRE rather than the JDK. Therefore, I needed to specify the JDK source zip file for the JREs as well.
