Saint's Log

Bash Scripting Newbie Notes

Last month I took on the task of writing a script to run the JDK JMH microbenchmarks. One of my coworkers already had a bash script for running a different set of benchmarks (GC), so I used it as a starting point. Having written mostly C# code in that past few years, I found the bash scripting environment unintuitive. Thankfully, there is a bash manual to refer to. I thought it might be worthwhile documenting a few of my lessons and surprises from that undertaking.

Bash Ignores +x File Mode

As a mostly-Windows user, I am accustomed to batch files (.bat) and PowerShell scripts (.ps1). These do not need any ceremony to declare them executable. I was using Windows 11 and testing my scripts in a bash environment in Windows Terminal. However, the scripts didn’t work when cloned onto my Linux and MacOS systems. That was when I observed that such scripts are not shown as executable by ls -l. However, bash runs them just fine in the Windows Terminal as shown below (note that this is the version of bash distributed with git-scm (and therefore based on MSYS)!

$ echo "uname -a" > test.sh; ls -l; ./test.sh
total 1
-rw-r--r-- 1 saint 1049089 9 Jan  8 15:43 test.sh
MSYS_NT-10.0-22000 SAINT 3.1.7-340.x86_64 2020-10-23 13:08 UTC x86_64 Msys

Interestingly, chmod cannot set the execute mode in the Windows Terminal.

$ chmod +x ./test.sh; ls -l
total 1
-rw-r--r-- 1 saint 1049089 9 Jan  8 15:43 test.sh

However, prefixing the script with #!/bin/bash makes ls -l show it as executable without any other changes.

$ echo -e '#!/bin/bash\nuname -a' > test.sh; ls -l
total 1
-rwxr-xr-x 1 saint 1049089 21 Jan  8 16:16 test.sh*

On MacOS (and Linux), the script cannot be executed without running chmod +x ./test.sh

% echo "uname -a" > test.sh; ls -l; ./test.sh
total 8
-rw-r--r--  1 saint  staff  9 Jan  8 15:49 test.sh
zsh: permission denied: ./test.sh

Also note that the #!/bin/bash prefix is required for command completion in the shell.

Echoing #!/bin/bash to a File Needs Single Quotes

Most of the script writing I did used double quotes (since they support variable interpolation, which I needed). I was testing the impact of not using the #!/bin/bash prefix in my scripts for the scenarios above when I ran into this strange error:

$ echo "#!/bin/bash" > temp.sh
bash: !/bin/bash: event not found

I had forgotten about the shell’s support for history expansion as pointed out in the solution here: the way around this is to use single quotes as in echo -e '#!/bin/bash\nuname -a'

… Bash Supports History Expansion

I played around with this history expansion feature to get a feel for it. I first renamed my .bash_history file then opened a new bash tab to start a new session with clean history.

uname -a
ls ~/.bash*
git --version
history
grep --version
which bash
!-2 # Shows grep version
!3  # Shows git version
!-5 # Shows history
!!  # Also shows history

Echoing Newlines Needs a Flag

The -e flag option is required to write newline characters to the output (instead of the literal \ and n characters. As per “Simple Commands” in the GNU bash manual, if the -e option is given, interpretation of the following backslash-escaped characters is enabled… \a, \b, \c, \e, \E, \f, \n, \r, \t, \v, \\, etc.

Git Supports File Modes

After pulling some changes to my MacBook and making some edits, I saw the diff below. I have never really had to pay attention to file modes, so this was intriguing.

diff --git a/java/jmh/setup_jmh_jdk_micros.sh b/java/jmh/setup_jmh_jdk_micros.sh
old mode 100644
new mode 100755

The difference is the execute bit is now on for all 3 user types. Apparently, 644 and 755 are the only file modes supported by git and the core.fileMode option, which tells Git if the executable bit of files in the working tree is to be honored, should be set to false if the filesystem a file is checked out on loses the executable bit on checkout. Open questions: does NTFS lose executable bits, i.e. does this property apply to NTFS? Is there a way to see the filemode in GitHub?

[ is an Executable!

Not even sure what to say about this one :D.

$ which [
/usr/bin/[

In Cygwin, we can get the path to the executable!

$ cygpath -w `which [`
C:\dev\cygwin64\bin\[.exe

Open questions: is this program being invoked for if-statements?

Miscellaneous Facts

These are observations are documented the Bash Manual but listed here since they’re things I learned along the way.

1. To view the full path of files in the current directory, use ls -d $PWD/*
2. To get rid of ^M (carriage-return characters) in a text file from Windows, open the file in vim then these commands: :e ++ff=dos then :set ff=unix then finally exit vim with :wq
3. $_ expands to the number of positional parameters. See Shell Parameters and Special Parameters.
4. $@ expands to the positional parameters, starting from one. It can be used to create an array of the parameters.
5. Argument numbering (aka positional parameters) is local to functions. In other words, when a function is executed, the arguments to the function become the positional parameters during its execution.
6. Functions cannot return any value, they return codes. Specifically, (as per the GNU bash manual’s Shell Functions section), if a numeric argument is given to return, that is the function’s return status; otherwise the function’s return status is the exit status of the last command executed before the return.
7. ~= is a binary operator used for regular expression matching. The string to the right of the operator is considered a POSIX extended regular expression. The return value is 0 if the string matches the pattern, and 1 otherwise. If the regular expression is syntactically incorrect, the conditional expression’s return value is 2. See the Conditional Constructs section of the GNU bash manual.

Open questions: how should the pattern in the JMH script be assigned to a variable?

Hints

See Windows Terminal tips and tricks for various useful customizations of the Windows Terminal. Changing tab colors is something I wouldn’t have thought to do but now makes it very easy to know which tabs are relevant for the task at hand.