Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker development environment does not work (Ubuntu 24.04.1), cannot find all repos. #1254

Open
Syonyk opened this issue Jan 2, 2025 · 11 comments

Comments

@Syonyk
Copy link
Contributor

Syonyk commented Jan 2, 2025

I'm attempting to use the Docker development environment on Ubuntu 24.04, and it does not properly update the environment. The issue seems to be a change in a path for Debian security repos.

Error log:

Image older than a week, Rebuild? [Y/n]: Y
[+] Building 4.2s (9/20)                                         docker:default
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 4.77kB                                     0.0s
 => [internal] load metadata for docker.io/library/debian:testing-slim     0.9s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 72B                                           0.0s
 => [ 1/16] FROM docker.io/library/debian:testing-slim@sha256:2ed89b1e801  0.0s
 => [internal] load build context                                          0.0s
 => => transferring context: 264B                                          0.0s
 => CACHED [ 2/16] RUN   apt-get update -y &&   apt-get upgrade -y &&   a  0.0s
 => CACHED [ 3/16] COPY docker/bin /tmp/simde-bin                          0.0s
 => CACHED [ 4/16] RUN   for script in simde-reset-build.sh; do     ln -s  0.0s
 => ERROR [ 5/16] RUN   apt-get update -y &&   apt-get upgrade -y &&   ap  3.2s
------                                                                          
 > [ 5/16] RUN   apt-get update -y &&   apt-get upgrade -y &&   apt-get install -yq --no-install-recommends     git build-essential vim     ccache cmake ninja-build     '^clang-[0-9\.]+$'     '^llvm-[0-9\.]+$'     '^g(cc|\+\+)-[0-9\.]+$'     gdb valgrind     binfmt-support qemu-user-static     creduce screen htop parallel nano rsync strace     npm libsleef-dev     pipx wget curl:                  
0.237 Get:1 http://mirror.it.ubc.ca/debian stable InRelease [151 kB]
0.355 Get:2 http://deb.debian.org/debian testing InRelease [175 kB]
0.376 Ign:3 http://security.debian.org stable/updates InRelease
0.386 Get:4 http://mirror.it.ubc.ca/debian stable/contrib amd64 Packages [54.1 kB]
0.431 Get:5 http://mirror.it.ubc.ca/debian stable/main amd64 Packages [8789 kB]
0.666 Get:6 http://deb.debian.org/debian testing-updates InRelease [49.6 kB]
0.692 Err:7 http://security.debian.org stable/updates Release
0.692   404  Not Found [IP: 151.101.194.132 80]
0.854 Get:8 http://deb.debian.org/debian-security testing-security InRelease [43.5 kB]
0.881 Get:9 http://deb.debian.org/debian testing/main amd64 Packages.diff/Index [27.9 kB]
1.060 Get:10 http://deb.debian.org/debian testing/main amd64 Packages 2025-01-01-0208.29.pdiff [114 kB]
1.285 Get:11 http://mirror.it.ubc.ca/debian stable/non-free amd64 Packages [97.3 kB]
1.295 Get:12 http://deb.debian.org/debian testing/main amd64 Packages 2025-01-01-0804.38.pdiff [696 B]
1.430 Get:13 http://deb.debian.org/debian testing/main amd64 Packages 2025-01-01-1406.38.pdiff [1679 B]
1.565 Get:14 http://deb.debian.org/debian testing/main amd64 Packages 2025-01-01-2008.05.pdiff [5073 B]
1.706 Get:15 http://deb.debian.org/debian testing/main amd64 Packages 2025-01-02-0205.18.pdiff [39.2 kB]
1.842 Get:16 http://deb.debian.org/debian testing/main amd64 Packages 2025-01-02-1406.26.pdiff [470 B]
1.848 Get:16 http://deb.debian.org/debian testing/main amd64 Packages 2025-01-02-1406.26.pdiff [470 B]
2.389 Reading package lists...
3.205 E: The repository 'http://security.debian.org stable/updates Release' does not have a Release file.
------
Dockerfile:25
--------------------
  24 |     # Common packages
  25 | >>> RUN \
  26 | >>>   apt-get update -y && \
  27 | >>>   apt-get upgrade -y && \
  28 | >>>   apt-get install -yq --no-install-recommends \
  29 | >>>     git build-essential vim \
  30 | >>>     ccache cmake ninja-build \
  31 | >>>     '^clang-[0-9\.]+$' \
  32 | >>>     '^llvm-[0-9\.]+$' \
  33 | >>>     '^g(cc|\+\+)-[0-9\.]+$' \
  34 | >>>     gdb valgrind \
  35 | >>>     binfmt-support qemu-user-static \
  36 | >>>     creduce screen htop parallel nano rsync strace \
  37 | >>>     npm libsleef-dev \
  38 | >>>     pipx wget curl
  39 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c apt-get update -y &&   apt-get upgrade -y &&   apt-get install -yq --no-install-recommends     git build-essential vim     ccache cmake ninja-build     '^clang-[0-9\\.]+$'     '^llvm-[0-9\\.]+$'     '^g(cc|\\+\\+)-[0-9\\.]+$'     gdb valgrind     binfmt-support qemu-user-static     creduce screen htop parallel nano rsync strace     npm libsleef-dev     pipx wget curl" did not complete successfully: exit code: 100

The primary issue seems to be the Debian security repo:

3.205 E: The repository 'http://security.debian.org stable/updates Release' does not have a Release file.

Per https://serverfault.com/questions/1074688/security-debian-org-does-not-have-a-release-file-on-with-debian-docker-images the path has changed from deb http://security.debian.org/debian-security stable/updates main to deb http://security.debian.org/debian-security stable-security/updates main - and this is also mentioned in a note in the Debian handbook: https://debian-handbook.info/browse/en-US/stable/apt.html#sect.apt-sources.list.testing

I'm not familiar enough with Docker to know how to go about fixing this, but if someone can point me in the right direction, I'm happy to create a PR to fix it.

@mr-c
Copy link
Collaborator

mr-c commented Jan 2, 2025

Hello @Syonyk ; can you try docker pull docker.io/library/debian:testing-slim and then re-run the docker/simde-dev.sh script?

@Syonyk
Copy link
Contributor Author

Syonyk commented Jan 2, 2025

It took longer to fail, but appears to have failed on a similar issue later on.

desktop:~/Dev/simde/docker$ sudo docker pull docker.io/library/debian:testing-slim
testing-slim: Pulling from library/debian
dc55741fa17c: Already exists 
Digest: sha256:2ed89b1e8012d945cfcc111fa1dc11a628edaa24b9af5d63d6935b5ee35d3377
Status: Downloaded newer image for debian:testing-slim
docker.io/library/debian:testing-slim

desktop:~/Dev/simde/docker$ sudo ./simde-dev.sh 
Image older than a week, Rebuild? [Y/n]: Y
[+] Building 52.8s (9/20)                                docker:default
 => [internal] load build definition from Dockerfile               0.0s
 => => transferring dockerfile: 4.77kB                             0.0s
 => [internal] load metadata for docker.io/library/debian:testing  0.0s
 => [internal] load .dockerignore                                  0.0s
 => => transferring context: 72B                                   0.0s
 => CACHED [ 1/16] FROM docker.io/library/debian:testing-slim      0.0s
 => [internal] load build context                                  0.0s
 => => transferring context: 264B                                  0.0s
 => [ 2/16] RUN   apt-get update -y &&   apt-get upgrade -y &&    49.8s
 => [ 3/16] COPY docker/bin /tmp/simde-bin                         0.0s
 => [ 4/16] RUN   for script in simde-reset-build.sh; do     ln -  0.1s
 => ERROR [ 5/16] RUN   apt-get update -y &&   apt-get upgrade -y  2.8s 
------                                                                  
 > [ 5/16] RUN   apt-get update -y &&   apt-get upgrade -y &&   apt-get install -yq --no-install-recommends     git build-essential vim     ccache cmake ninja-build     '^clang-[0-9\.]+$'     '^llvm-[0-9\.]+$'     '^g(cc|\+\+)-[0-9\.]+$'     gdb valgrind     binfmt-support qemu-user-static     creduce screen htop parallel nano rsync strace     npm libsleef-dev     pipx wget curl:
0.253 Get:1 http://mirror.it.ubc.ca/debian stable InRelease [151 kB]
0.294 Hit:2 http://deb.debian.org/debian testing InRelease
0.399 Get:3 http://mirror.it.ubc.ca/debian stable/contrib amd64 Packages [54.1 kB]
0.445 Get:4 http://mirror.it.ubc.ca/debian stable/non-free amd64 Packages [97.3 kB]
0.460 Get:5 http://mirror.it.ubc.ca/debian stable/main amd64 Packages [8789 kB]
0.468 Hit:6 http://deb.debian.org/debian testing-updates InRelease
0.492 Hit:7 http://deb.debian.org/debian-security testing-security InRelease
0.548 Ign:8 http://security.debian.org stable/updates InRelease
0.865 Err:9 http://security.debian.org stable/updates Release
0.865   404  Not Found [IP: 151.101.2.132 80]
1.932 Reading package lists...
2.718 E: The repository 'http://security.debian.org stable/updates Release' does not have a Release file.
------
Dockerfile:25
--------------------
  24 |     # Common packages
  25 | >>> RUN \
  26 | >>>   apt-get update -y && \
  27 | >>>   apt-get upgrade -y && \
  28 | >>>   apt-get install -yq --no-install-recommends \
  29 | >>>     git build-essential vim \
  30 | >>>     ccache cmake ninja-build \
  31 | >>>     '^clang-[0-9\.]+$' \
  32 | >>>     '^llvm-[0-9\.]+$' \
  33 | >>>     '^g(cc|\+\+)-[0-9\.]+$' \
  34 | >>>     gdb valgrind \
  35 | >>>     binfmt-support qemu-user-static \
  36 | >>>     creduce screen htop parallel nano rsync strace \
  37 | >>>     npm libsleef-dev \
  38 | >>>     pipx wget curl
  39 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c apt-get update -y &&   apt-get upgrade -y &&   apt-get install -yq --no-install-recommends     git build-essential vim     ccache cmake ninja-build     '^clang-[0-9\\.]+$'     '^llvm-[0-9\\.]+$'     '^g(cc|\\+\\+)-[0-9\\.]+$'     gdb valgrind     binfmt-support qemu-user-static     creduce screen htop parallel nano rsync strace     npm libsleef-dev     pipx wget curl" did not complete successfully: exit code: 100

@mr-c
Copy link
Collaborator

mr-c commented Jan 2, 2025

@Syonyk Maybe 0.865 404 Not Found [IP: 151.101.2.132 80] is a transient error? Can you try a couple more times?

Perhaps there is a networking / VPN or firewall issue

@Syonyk
Copy link
Contributor Author

Syonyk commented Jan 2, 2025

I don't believe it's a transient issue.

There is no http://security.debian.org/dists/stable/ path - take a look at http://security.debian.org/dists/. It should be using http://security.debian.org/dists/stable-security/

I don't know enough about Docker to know where that out of date apt sources.list file is coming from.

@mr-c
Copy link
Collaborator

mr-c commented Jan 2, 2025

Odd, it works for me using podman; here's the log after clearing my cache:

$ docker/simde-dev.sh 
Image older than a week, Rebuild? [Y/n]: y
STEP 1/21: FROM debian:testing-slim
Resolved "debian" as an alias (/etc/containers/registries.conf.d/shortnames.conf)
Trying to pull docker.io/library/debian:testing-slim...
Getting image source signatures
Copying blob dc55741fa17c done  
Copying config 15e9e3d69d done  
Writing manifest to image destination
Storing signatures
STEP 2/21: ARG DEBIAN_FRONTEND=noninteractive
--> 65a7d77a1c5
STEP 3/21: ARG QEMU_GIT=n
--> f4c3d5142fb
STEP 4/21: ARG ARCHS="arm64 armel armhf i386 mips64el ppc64el s390x riscv64"
--> 6b2ce0c5b06
STEP 5/21: RUN   apt-get update -y &&   apt-get upgrade -y &&   apt-get install -y netselect-apt &&   if netselect-apt -n ${release}; then     mv sources.list /etc/apt/sources.list;   fi
Get:1 http://deb.debian.org/debian testing InRelease [175 kB]
Get:2 http://deb.debian.org/debian testing-updates InRelease [49.6 kB]
Get:3 http://deb.debian.org/debian-security testing-security InRelease [43.5 kB]
Get:4 http://deb.debian.org/debian testing/main amd64 Packages [9385 kB]
Fetched 9653 kB in 2s (5424 kB/s)

@mr-c
Copy link
Collaborator

mr-c commented Jan 2, 2025

I got the same result using the official docker packages. Maybe check the instructions at https://docs.docker.com/engine/install/ubuntu/ ?

$ docker version
Client: Docker Engine - Community
 Version:           27.4.1
 API version:       1.47
 Go version:        go1.22.10
 Git commit:        b9d17ea
 Built:             Tue Dec 17 15:45:56 2024
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          27.4.1
  API version:      1.47 (minimum version 1.24)
  Go version:       go1.22.10
  Git commit:       c710b88
  Built:            Tue Dec 17 15:45:56 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.7.24
  GitCommit:        88bf19b2105c8b17560993bee28a01ddc2f97182
 runc:
  Version:          1.2.2
  GitCommit:        v1.2.2-0-g7cb3632
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

@mr-c
Copy link
Collaborator

mr-c commented Jan 2, 2025

whoops, I take that back, with the official docker packages I can reproduce your error. I suggest switching to podman :-)

@mr-c
Copy link
Collaborator

mr-c commented Jan 2, 2025

I think it is the netselect-apt part isn't working. I'm testing without that now

diff --git a/docker/Dockerfile b/docker/Dockerfile
index 70d91b77..d1f83a92 100644
--- a/docker/Dockerfile
+++ b/docker/Dockerfile
@@ -7,14 +7,6 @@ ARG DEBIAN_FRONTEND=noninteractive
 ARG QEMU_GIT=n
 ARG ARCHS="arm64 armel armhf i386 mips64el ppc64el s390x riscv64"
 
-RUN \
-  apt-get update -y && \
-  apt-get upgrade -y && \
-  apt-get install -y netselect-apt && \
-  if netselect-apt -n ${release}; then \
-    mv sources.list /etc/apt/sources.list; \
-  fi
-
 COPY docker/bin /tmp/simde-bin
 RUN \
   for script in simde-reset-build.sh; do \

@mr-c
Copy link
Collaborator

mr-c commented Jan 2, 2025

Yes, netselect is to blame: https://bugs.debian.org/1012313

@Syonyk
Copy link
Contributor Author

Syonyk commented Jan 2, 2025

Is there an easy fix to not use it/work around it? I'll try podman, though.

@mr-c
Copy link
Collaborator

mr-c commented Jan 2, 2025

Is there an easy fix to not use it/work around it? I'll try podman, though.

Remove the netselect lines as shown in #1254 (comment) ; that worked for me using the official docker package. You might enjoy podman for other reasons, but it turns out that it is not needed after all

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants