Categories: Containers

Installing Docker on Ubuntu

I was looking for the authoritative way to install docker on Ubuntu. install docker ubuntu – Google Search points me to Install Docker Engine on Ubuntu | Docker Docs. Running this command shows that none of the packages in the Uninstall old versions section are installed on my Ubuntu VM.

for pkg in docker.io docker-doc docker-compose podman-docker containerd runc; do sudo apt-get remove $pkg; done

Docker Engine comes bundled with Docker Desktop for Linux. This is the easiest and quickest way to get started.

Install Docker Engine on Ubuntu | Docker Docs

The Docker Desktop generic installation steps link to Install Docker Desktop on Ubuntu | Docker Docs. Step 1 is to set up Docker’s package repository.

# Add Docker's official GPG key:
sudo apt-get update
sudo apt-get install ca-certificates curl gnupg
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg

# Add the repository to Apt sources:
echo \
  "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update

I end up running step 2 as well to install the docker engine and call it good.

sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Skipping the package repository setup step will result in these errors (seen on x64 5.10.102.1-microsoft-standard-WSL2 but all other steps and output are from a VM):

Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Package docker-ce is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'docker-ce' has no installation candidate
E: Unable to locate package docker-ce-cli
E: Unable to locate package containerd.io
E: Couldn't find any package by glob 'containerd.io'
E: Couldn't find any package by regex 'containerd.io'
E: Unable to locate package docker-buildx-plugin
E: Unable to locate package docker-compose-plugin

I list the available containers by running docker ps and there are none, but this verifies that docker is working.

saint@ubuntuvm:~$ sudo docker ps
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
saint@ubuntuvm:~$

The hello-world image runs successfully as well.

saint@ubuntuvm:~$ sudo docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
719385e32844: Pull complete 
Digest: sha256:88ec0acaa3ec199d3b7eaf73588f4518c25f9d34f58ce9a0df68429c5af48e8d
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (amd64)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/get-started/

saint@ubuntuvm:~$ 

Running Docker in WSL

I followed the above steps to install docker in my Windows Subsystem for Linux Ubuntu 22.04.2 LTS environment. Unfortunately, docker ps does not work.

saint@mymachine:~$ sudo docker ps
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
saint@mymachine:~$ 

linux – Docker not running on Ubuntu WSL due to error cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running? – Stack Overflow suggests running sudo dockerd. Here is the tail end of the output, including an error.

ERRO[2023-10-17T09:09:19.059240012-06:00] failed to initialize a tracing processor "otlp"  error="no OpenTelemetry endpoint: skip plugin"
INFO[2023-10-17T09:09:19.059460691-06:00] serving...                                    address=/var/run/docker/containerd/containerd-debug.sock
INFO[2023-10-17T09:09:19.059530687-06:00] serving...                                    address=/var/run/docker/containerd/containerd.sock.ttrpc
INFO[2023-10-17T09:09:19.059629051-06:00] serving...                                    address=/var/run/docker/containerd/containerd.sock
INFO[2023-10-17T09:09:19.059665540-06:00] containerd successfully booted in 0.025117s
INFO[2023-10-17T09:09:19.114570236-06:00] [graphdriver] using prior storage driver: overlay2
INFO[2023-10-17T09:09:19.114803099-06:00] Loading containers: start.
INFO[2023-10-17T09:09:19.297993571-06:00] stopping event stream following graceful shutdown  error="<nil>" module=libcontainerd namespace=moby
INFO[2023-10-17T09:09:19.298958219-06:00] stopping healthcheck following graceful shutdown  module=libcontainerd
INFO[2023-10-17T09:09:19.299104948-06:00] stopping event stream following graceful shutdown  error="context canceled" module=libcontainerd namespace=plugins.moby
failed to start daemon: Error initializing network controller: error obtaining controller instance: unable to add return rule in DOCKER-ISOLATION-STAGE-1 chain:  (iptables failed: iptables --wait -A DOCKER-ISOLATION-STAGE-1 -j RETURN: iptables v1.8.7 (nf_tables):  RULE_APPEND failed (No such file or directory): rule in chain DOCKER-ISOLATION-STAGE-1
 (exit status 4))

I start by searching for the first error, failed to initialize a tracing processor “otlp” error=”no OpenTelemetry endpoint: skip plugin” – Google Search, instead of the last. However, I find Failed to start docker on WSL · Issue #8450 · microsoft/WSL (github.com) and it has the solution:

edit /etc/default/docker and add DOCKER_OPTS="--iptables=false"

Failed to start docker on WSL · Issue #8450 · microsoft/WSL (github.com)

Sharing Files with Ubuntu Guest on Hyper-V Host

Of the many ways to transfer files to an Ubuntu guest on Hyper-V, running these PowerShell commands (as admin) suffices for a one-off file transfer. See 4 Ways to Transfer Files to a Linux Hyper-V Guest (altaro.com) for more details about this approach.

Enable-VMIntegrationService -VMName 'Ubuntu 22.04 LTS' -Name 'Guest Service Interface'

Copy-VMFile -Name 'Ubuntu 22.04 LTS' -SourcePath 'dumpfile.gz' -DestinationPath '/home/saint/Downloads' -FileSource Host
Copy-VMFile in Action

Backstory

Yesterday I had a core dump from a Linux process that I wanted to specifically inspect in an Ubuntu VM. My host machine is a Windows 11 (10.0.22621.674) machine. The simple question of how to share files with my Ubuntu VM took me all over the map. Searching for hyper-v share files linux guest led me to Shared Folders over Hyper-V Ubuntu Guest (linuxhint.com). This had me enabling SMB 1.0/CIFS File Sharing Support (already had SMB Direct enabled) and Public folder sharing.

SMB Windows Features
Public Folder Sharing Settings

I then created an empty directory and turned on sharing on it as instructed. However, accessing it from Ubuntu turned out to be the problem. These are the suggested commands:

sudo apt install cifs-utils
mkdir ~/SharedFolder
sudo mount.cifs //<NAME OF YOUR PC>/<SHARED FOLDER NAME>
~/SharedFolder -o user=<YOUR WINDOWS USERNAME>

mount.cifs failed though.

saint@linuxvm:~$ sudo mount.cifs //DEVICENAME/virtual-machines
~/shared -o user=USERNAME
Password for USERNAME@//DEVICENAME/virtual-machines: ***
mount error(13): Permission denied
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs) and kernel log messages (dmesg)

There doesn’t seem to be anything particularly interesting at mount.cifs(8) – Linux man page (die.net). Running dmesg showed these messages:

[  425.318905] CIFS: Attempting to mount \\DEVICENAME\virtual-machines
[  425.318905] CIFS: Status code returned 0xc000006d STATUS_LOGON_FAILURE
[  425.318905] CIFS: VFS: \\DEVICENAME Send error in SessSetup = -13
[  425.318905] CIFS: VFS: cifs_mount failed w/return code = -13

cifs status_logon_failure – Search (bing.com) leads to a comment at STATUS_LOGON_FAILURE (0xc000006d) · Issue #478 · hierynomus/smbj (github.com) stating that STATUS_LOGON_FAILURE means that your credentials were rejected. This error code (and others) at [MS-CIFS]: SMB Error Classes and Codes | Microsoft Learn. The Windows event logs do not contain any entries related to this (surprisingly). So I pivot to the next result from my search for hyper-v share files linux guest.

4 Ways to Transfer Files to a Linux Hyper-V Guest (altaro.com) instructs you to enable the file copy guest service (either using PowerShell or the GUI). Apparently a power cycle of the VM is not necessary. See article for more info.

Enable-VMIntegrationService -VMName LinuxVM3 -Name 'Guest Service Interface'

Copy-VMFile -Name LinuxVM3 -SourcePath 'dumpfile.gz' -DestinationPath '/home/saint/Downloads' -FileSource Host

Unfortunately, Copy-VMFile fails. The VM is running Ubuntu 20.04.1 (x86_64) with kernel 5.15.0-52-generic.

It shouldn’t be this hard to just get a file into a guest VM. Looking up the docs again and Use local resources on Hyper-V virtual machine with VMConnect | Microsoft Learn suggests VMConnect but looks like enhanced session mode and Type Clipboard text are only available on VMs running a recent Windows OS. For Ubuntu, that article points to Changing Ubuntu Screen Resolution in a Hyper-V VM | Microsoft Learn. At this point, I decide to create a new VM using Hyper-V’s quick create and perhaps that will have the proper configuration for what I’m trying to do.

Creating an Ubuntu VM

Click on Hyper-V’s Quick Create… command to start creating a VM. Select the latest Ubuntu LTS (22.04). Unfortunately, the only options available are the VM name and the network switch to use. Clicking on Create Virtual machine creates a VM on the primary/OS disk. I was pleasantly surprised to find that the Ubuntu 22.04 VM appeared to support enhanced session mode when Hyper-V asked for the screen resolution when connecting to it:

Connecting to Ubuntu VM

The enhanced session gives this xrdp login window:

xrdp Login Window

The window disappears when I enter my credentials and nothing happens for some time. I used the “Basic Session” toolbar button to switch back to the normal mode I’m used to. These are some of the errors I encounter:

Oh no! Something has gone wrong.
Internal Error Details

The error report points out that I have obsolete packages, among them gnome-shell (which crashed). I run sudo apt upgrade and says yes to the 368 upgrades (826 MB of archives). That is not sufficient to address this rdp bug so I stay in Basic Session mode for the rest of the time.

This leads me back to the PowerShell commands I used above. Lo and behold, they work this time! This is despite the fact that there don’t appear to be any processes displayed by ps -u root | grep hyper as described at 4 Ways to Transfer Files to a Linux Hyper-V Guest (altaro.com).

Enable-VMIntegrationService -VMName 'Ubuntu 22.04 LTS' -Name 'Guest Service Interface'

Copy-VMFile -Name 'Ubuntu 22.04 LTS' -SourcePath 'dumpfile.gz' -DestinationPath '/home/saint/Downloads' -FileSource Host

This is when I discover that I do not have enough space on the VM to expand my .gz file.

Inspecting Disk Usage

Unfortunately, the disk for the VM is only 12 GB (confirmed by launching Ubuntu and running out of space). Therefore, once the installation completes, expand the disk from 12 GB to a more reasonable size (e.g. 127 GB). If the default drive Quick Create used for the VM’s virtual disk does not have sufficient space, you will need to move the virtual hard disk to another drive then expand the partition in Ubuntu to use the whole virtual disk.

Moving Ubuntu VM to a Bigger Disk

My main desktop has a 500 GB SSD that had only about 20GB of space free. How unpleasant to then discover that Quick Create simply dumped the new VM on it AND created such a small disk to start with, all without asking. Turns out I’m not the only one that finds this behavior less than ideal: hyperv quick create disk size – Search (bing.com) pointed me to Hyper-V Ubuntu 18.04 Quick Create disk size is too small · Issue #82 · microsoft/linux-vm-tools (github.com) and unfortunately, doesn’t look like there’s a resolution of this issue. My solution was to create a new virtual disk on my secondary 3.5 TB hard drive.

If the VM was still running, this error dialog will most likely be displayed.

After starting the VM again, I still didn’t have enough space to decompress my .gz file.

Inspecting Disk Usage

Fortunately, there is a useful site explaining how to Expand Ubuntu disk after Hyper-V Quick Create – Anton Karl Ingason (linguist.is):

sudo apt install cloud-guest-utils
sudo growpart /dev/sda 1
sudo resize2fs /dev/sda1

growpart failed the first time I ran it. The disk was still 12 GB!

I had to turn off the VM, wait for the disk “merging” status to go away, then go to edit the disk in Hyper-V:

Some scary warnings about data loss that I promptly ignored and marched forward since I didn’t yet have any critical data on that disk.

Once the expansion completes, the growpart command can now be successfully exeuted as shown below.

Running growpart in Ubuntu

Open Questions

  1. Why does mount.cifs fail (on both VMs)?
  2. Why does Copy-VMFile work on Ubuntu 22 VM but not Ubuntu 20?