Compilers – Page 2 – Saint's Log

2021-11-29 —Categories: Compilers

Ubuntu VM Setup for OpenJDK Development

I’m using a Windows 10 physical machine for my OpenJDK 17 development. Unfortunately, I ran into some issues getting the environment set up to build the JDK on Windows. To work around this, I created a Linux virtual machine. Although the instructions for building on Linux are on the OpenJDK site, I would like to have all the instructions in one spot, hence this post.

Creating an Ubuntu VM in Hyper-V

Download an LTS Ubuntu .iso from the Ubuntu Desktop download page. I selected Ubuntu 20.04.3 LTS.
Go to New > Virtual Machine in Hyper-V manager.
1. Enter your VM name, generation, memory amount and type
2. Select the connection type (Default Switch) and create a new virtual hard disk
3. Select “Install an operating system from a bootable CD/DVD-ROM” then enter the path to the downloaded .iso file then click on Finish.
4. Before starting the VM, set the number of virtual processors (it defaults to 1, which is less than ideal)!
Perform a normal Ubuntu installation including erasing the disk

Let us now review the more interesting steps – those related to configuring the Ubuntu environment.

Increase the Resolution of the Ubuntu Guest OS

The default 1024×768 screen resolution of the Ubuntu guest is rather restrictive. The solution to this comes from https://askubuntu.com/questions/384602/ubuntu-hyper-v-guest-display-resolution. We need to configure the Hyper-V Synthetic Video Frame Buffer Driver by adding ” video=hyperv_fb:1680×1050” to the GRUB_CMDLINE_LINUX_DEFAULT value in the /etc/default/grub file.

sudo apt-get install linux-image-extra-virtual
sudo apt-get install vim
sudo vim /etc/default/grub
sudo update-grub
reboot

Install the development dependencies

The table below lists the JDK build dependencies and the commands to install them.

Component	Installation Command
autoconf	sudo apt-get install autoconf
Git	sudo apt-get install git
C Compiler	sudo apt-get install build-essential
X11 libraries	sudo apt-get install libx11-dev libxext-dev libxrender-dev libxrandr-dev libxtst-dev libxt-dev
cups	sudo apt-get install libcups2-dev
fontconfig	sudo apt-get install libfontconfig1-dev
alsa	sudo apt-get install libasound2-dev

JDK Build Dependencies

This single command suffices to install all these components.

sudo apt-get install autoconf git build-essential libx11-dev libxext-dev libxrender-dev libxrandr-dev libxtst-dev libxt-dev libcups2-dev libfontconfig1-dev libasound2-dev

Install a code editor

Download the Visual Studio Code .deb file from https://code.visualstudio.com/Download. We can then install VS Code by running:

sudo apt install ~/Downloads/code_1.62.3-1637137107

Install a Boot JDK

I use the Microsoft OpenJDK build as the boot JDK. Here are the Ubuntu instructions for Installing the Microsoft Build of OpenJDK:

# Valid values are only '18.04' and '20.04'
# For other versions of Ubuntu, please use the tar.gz package
ubuntu_release=`lsb_release -rs`
cd ~/Downloads/
wget https://packages.microsoft.com/config/ubuntu/${ubuntu_release}/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
sudo dpkg -i packages-microsoft-prod.deb
sudo apt-get install apt-transport-https

sudo apt-get update
sudo apt-get install msopenjdk-17

Verify that everything is working by running “java -version”

Clone and Build the JDK

Clone the JDK. Note that cloning a fork might be much slower than cloning the upstream Github repo! I was averaging about 60KiB/s on my rork whereas cloning the upstream OpenJDK was averaging 6 MiB/s when receiving objects!

mkdir ~/repos
cd ~/repos
git clone https://github.com/openjdk/jdk

The JDK repo can now be configured and built

cd jdk
bash configure
make images

The configure command should display any missing dependencies that it needs and a suggestion for how to install them.

To try out your new build, switch to the bin folder and check the Java version:

cd ~/repos/jdk/build/linux-x86_64-server-release/jdk/bin
./java -version

To browse through the contents of the build folder in a file manager:

xdg-open ./build

2021-11-22 —Categories: Compilers

Entering the Compiler Space

Last week was my first week in the Java engineering group. It has been about 11 years since I took a compiler course (while in the CS MS program at BYU). A quick review of the history of Java was in order. Turns out I last used Java in 2012 in grad school. That must have been Java SE 7 from 2011 and Java SE 6 before that. Since I have not been in the compiler space since then, I have a steep learning curve ahead. That is the exciting thing about technology though – there is always more to learn!

I am currently a programmer in the developer division at Microsoft so it was helpful going through some of the Java development with Microsoft documentation for a high level overview of all our offerings. Also informative given my long absence from Java-land were the docs on how to Transition from Java 7 to Java 8 and from Java 8 to Java 11. It hadn’t yet dawned on me by the time I read through these that the reason references to 8, 11, and 17 keep coming up is because they are LTS releases.

As a newbie to the Java development world, I started by watching this 2019 OpenJDK Development talk on how to become an OpenJDK contributor. It is a great overview of concepts like project roles (author, committer, reviewer, etc), the contributor agreement, and (perhaps most importantly to me), how to find an issue to work on and build the OpenJDK. The breakdown of commonly used terminology and abbreviations was great to have as well.

For an introduction to the hotspot compiler, I started going through “A Simple Graph-Based Intermediate Representation“. I ended up watching Cliff Click’s talk on The Sea of Nodes and the HotSpot JIT before I got that far along in the paper. It was fascinating seeing details such as the CPU L1/L2 cache size playing into the design! Some of the concepts that I need to review after that talk include:

SSA, Peephole optimizations, Register allocation
Java’s type system (e.g. what is the significance of lattices or Hash consing?)
Probably won’t hurt to comb through Engineering a Compiler as well.

The sea of nodes talk also revealed to me how little I know about companies in the Java space. I don’t think I had heard of Azul before, for example. In fact, it’s not just companies but also technologies! I was going through some build documentation when I ran into mentions of AdoptOpenJDK and Adoptium, both of which were foreign to me. I was glad though to see my old friend Eclipse doing well.

One of the most enjoyable things about being a programmer is working with very skilled people, especially watching them in action! I always learn a lot! My colleagues David and Mat were kind enough to pull me into their triage and reporting of [JDK-8277299] STACK_OVERFLOW in Java_sun_awt_shell_Win32ShellFolder2_getIconBits – Java Bug System so I could get my feet wet with how things are done in OpenJDK development.

The OpenJDK process is certainly different from the other open source communities I’ve been a part of (.NET and Mozilla Firefox). My manager and I poked around the bug DB to see what compiler starter bugs are out there. I picked bug [JDK-7077093] labelOper::label() should return Label& but since I must start out as an author, issues cannot be assigned to me. Unusual to me but the logic appears sound. Here is the query for C2 starter bugs.

Other highlights of the week were setting up my dev box to build the OpenJDK source code (unsuccessfully), discovering that compiler explorer is a thing (and an open source one at that), learning from my teammates how to investigate a failure of a fairly complex test on MacOS (they were using LLDB). I hope to write follow-up entries on these at some point.