Saint's Log

Building Info-ZIP Source Code

I have been working on building the OpenJDK for the Windows ARM64 platform. The make images command has been failing in Cygwin with errors such as:

Creating java.se.jmod
zip I/O error: Device or resource busy
zip error: Could not create output file (/cygdrive/d/dev/repos/jdk/build/windows-aarch64-server-release/support/src.zip)
make[4]: *** [ZipSource.gmk:79: /cygdrive/d/dev/repos/jdk/build/windows-aarch64-server-release/support/src.zip] Error 1
make[4]: *** Deleting file '/cygdrive/d/dev/repos/jdk/build/windows-aarch64-server-release/support/src.zip'
make[3]: *** [ZipSource.gmk:93: zip] Error 2
make[2]: *** [make/Main.gmk:389: zip-source] Error 2
make[2]: *** Waiting for unfinished jobs....

ERROR: Build failed for target 'images' in configuration 'windows-aarch64-server-release' (exit code 2)

One of the troubleshooting approaches I tried was to build an instrumented zip executable to replace Cygwin’s zip.exe. I started by searching for the Cygwin zip package. Someone was kind enough to have a link to the Cygwin zip package’s official page which in turn linked to the Cygwin zip packaging repository. This repo’s tree view shows a single file with source URIs for zip and that’s how I learned that this zip utility was Info-ZIP. The Zip 3.0 page has a link to Info-ZIP’s SourceForge site, from which the zip sources can be downloaded. I used curl in the Windows Terminal as follows:

curl -Lo zip30.tar.gz https://sourceforge.net/projects/infozip/files/Zip%203.x%20%28latest%29/3.0/zip30.tar.gz/download
tar xvf zip30.tar.gz
cd ./zip30
git init; git add *; git commit -m "Commit original Info-ZIP sources"

Now that we have the sources, let’s see how to build them. The scenario I’m working on is Windows specific so we need Visual Studio 2019 with the Desktop Development with C++ workload installed. I’ll be building a 32-bit zip executable. Launch the x86 Native Tools Command Prompt for VS 2019 and change to the zip30 source directory to start building. Some digging around reveals a makefile with build instructions (that seem one directory off). Here’s the command to build a 32-bit executable from the sources (note that building fails due to various errors that need to be addressed):

nmake -f win32\makefile.w32

Carriage Return (CR) Name Collisions

The first error is this rather cryptic mess of syntax errors:

Microsoft (R) Program Maintenance Utility Version 14.29.30133.0
Copyright (C) Microsoft Corporation.  All rights reserved.

        cl -nologo -c -W3 -O2 -DWIN32 -DASM_CRC -ML  zip.c
cl : Command line warning D9002 : ignoring unknown option '-ML'
zip.c
C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um\winnt.h(18822): error C2143: syntax error: missing ':' before 'constant'
C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um\winnt.h(18822): error C2143: syntax error: missing ';' before ':'
C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um\winnt.h(18822): error C2059: syntax error: ':'
C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um\winnt.h(18823): error C2143: syntax error: missing '{' before ':'
C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um\winnt.h(18823): error C2059: syntax error: ':'
C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um\winnt.h(18824): error C2059: syntax error: '}'
C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um\winnt.h(18825): error C2059: syntax error: '}'
C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um\winnt.h(18826): error C2059: syntax error: '}'
zip.c(5746): warning C4267: '=': conversion from 'size_t' to 'ush', possible loss of data
zip.c(5838): warning C4267: '=': conversion from 'size_t' to 'ush', possible loss of data
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\bin\HostX86\x86\cl.EXE"' : return code '0x2'
Stop.

Some head banging (and not looking at the precompiler output from cl -c -W3 -O2 -DWIN32 -DASM_CRC /P zip.c carefully) leads to c – Compiler errors in WINNT.H after retargeting solution to latest SDK (10.0.18362) – Stack Overflow where the solution is rather simple: do not use the name CR to define the carriage return since it maps to a bitfield in one of the structs in winnt.h. See Rename CR identifier to avoid collisions with ARM64 structs in winnt.h by swesonga · Pull Request #2 · swesonga/Info-ZIP (github.com)

Outdated Linker Flags

The next error is a complaint about the /OPT:NOWIN98 flag.

        link -nologo user32.lib advapi32.lib /OPT:NOWIN98 /INCREMENTAL:NO /PDB:zip.pdb  /RELEASE zip.obj crypt.obj ttyio.obj zipfile.obj zipup.obj fileio.obj util.obj  crc32.obj crci386c.obj globals.obj deflate.obj trees.obj match32.obj win32.obj win32zip.obj nt.obj win32i64.obj zip.res
LINK : fatal error LNK1117: syntax error in option 'OPT:NOWIN98'
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\bin\HostX86\x86\link.EXE"' : return code '0x45d'
Stop.

Turns out this option was removed in Visual Studio 2010 as per the Microsoft C/C++ change history since the linker no longer supports optimizing for Windows 98. This is clearly a safe flag to remove from the linker flags in win32\makefile.w32.

Update the Branding

1. Change the VERSION string from “3.0” to “3.0-ioHardenedZIP”
2. Update the REVDATE from “July 5th 2008” to the current date (“December 18th 2021” in my case)
3. Update the about text to indicate that it is a custom build.

Testing the Zip Build

The sources should now build successfully in the x86 Native Tools Command Prompt for VS 2019. The OpenJDK build uses the -qru flags for creating zip files so we can easily test the zip executable by creating a zip of the Info-ZIP help and license text.

zip -h > help.txt
zip -h2 > help2.txt
zip -L > license.txt
zip -qru ./files.zip -i *.txt

We need to verify whether the zip was correctly created. Saving this for another day.

Building on Linux/macOS

[Update 2021-12-22] To build Info-ZIP on macOS, the way the memset function is detected needs to be fixed. Info-ZIP for either Linux or macOS can then be built using this command:

make -f unix/Makefile generic