UndefinedVar triggered in heredoc code

[WanWizard]

Contributing guidelines

• I've read the contributing guidelines and wholeheartedly agree

I've found a bug and checked that ...

• ... the documentation does not mention anything about my problem
• ... there are no open or closed issues that are related to my problem

Description

Consider this Dockerfile snippet:

COPY <<EOF /etc/environment
export LC_ALL=en_US.utf8
export LANG=$LC_ALL
export LC_CTYPE=$LC_ALL
export LC_NUMERIC=$LC_ALL
export LC_TIME=$LC_ALL
export LC_COLLATE=$LC_ALL
export LC_MONETARY=$LC_ALL
export LC_MESSAGES=$LC_ALL
export LC_PAPER=$LC_ALL
export LC_NAME=$LC_ALL
export LC_ADDRESS=$LC_ALL
export LC_TELEPHONE=$LC_ALL
export LC_MEASUREMENT=$LC_ALL
export LC_IDENTIFICATION=$LC_ALL
EOF

This now prints

 1 warning found (use docker --debug to expand):
 - UndefinedVar: Usage of undefined variable '$LC_ALL' (line 59)

even though this isn't code related to building the docker image.

I know it is cosmetic and probably low prio, but it still annoys me every time I see it... 😁

Expected behaviour

No warnings printed.

Actual behaviour

Outputs

 1 warning found (use docker --debug to expand):
 - UndefinedVar: Usage of undefined variable '$LC_ALL' (line 59)

when building the docker image.

Buildx version

github.com/docker/buildx v0.31.1 a267595

Docker info

Client: Docker Engine - Community
 Version:    29.2.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.31.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v5.1.0
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 4
 Server Version: 29.2.1
 Storage Driver: overlay2
  Backing Filesystem: xfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 CDI spec directories:
  /etc/cdi
  /var/run/cdi
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: dea7da592f5d1d2b7755e3a161be07f43fad8f75
 runc version: v1.3.4-0-gd6d73eb8
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.14.0-611.34.1.el9_7.x86_64
 Operating System: AlmaLinux 9.7 (Moss Jungle Cat)
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 30.96GiB
 Name: batman.office.flexcoders.eu
 ID: 8b5933f9-3368-4c9c-a1eb-fefdc7153e93
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  ::1/128
  127.0.0.0/8
 Live Restore Enabled: false
 Firewall Backend: iptables+firewalld

Builders list

NAME/NODE     DRIVER/ENDPOINT   STATUS    BUILDKIT   PLATFORMS
default*      docker                                 
 \_ default    \_ default       running   v0.27.1    linux/amd64 (+3), linux/386

Configuration

See description.

Build logs

Additional info

No response

[thaJeztah]

I think this is the correct behavior (but can get confusing fast!); Your Dockerfile uses heredoc, but not a "quoted" word as terminator, which means that the Dockerfile frontend/parser will interpolate variables within the block before running;

See https://docs.docker.com/reference/dockerfile/#example-creating-inline-files

docker build --no-cache --progress=plain -<<'EOF'
# syntax=docker/dockerfile:1

FROM alpine
ENV HELLO=world
ARG HELLO_ARG=world

# interpolates
COPY <<EOT /a.txt
hello $HELLO
hello $HELLO_ARG
EOT

# does not interpolate
COPY <<'EOT' /b.txt
hello $HELLO
hello $HELLO_ARG
EOT

RUN cat /a.txt
RUN cat /b.txt

# caveat: interpolates at RUN time, not before, but still during "build":
RUN <<'EOT'
echo hello $HELLO
echo hello $HELLO_ARG
EOT

EOF

Output;

#11 [4/6] RUN cat /a.txt
#11 0.155 hello world
#11 0.155 hello world
#11 DONE 0.2s

#12 [5/6] RUN cat /b.txt
#12 0.159 hello $HELLO
#12 0.159 hello $HELLO_ARG
#12 DONE 0.2s

#13 [6/6] RUN <<'EOT' (echo hello world...)
#13 0.146 hello world
#13 0.146 hello world
#13 DONE 0.2s

[WanWizard]

Hmm, is this new? I can't recall seeing this when the Dockerfile was made (which I have to admit was a while ago).

Thanks for pointing this out! 👍

[WanWizard]

Hmm, looks like this is a case of either/or.

The complete snippet is

COPY <<EOF /etc/environment
export PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:${SCRIPTPATH}"

export LESS="-c -x4 -S -R"
export MINICOM="-c on"
export PAGER=/usr/bin/less
export TITLE="$(hostname)"
export VISUAL=/usr/bin/vim

export LC_ALL=en_US.utf8
export LANG=$LC_ALL
export LC_CTYPE=$LC_ALL
export LC_NUMERIC=$LC_ALL
export LC_TIME=$LC_ALL
export LC_COLLATE=$LC_ALL
export LC_MONETARY=$LC_ALL
export LC_MESSAGES=$LC_ALL
export LC_PAPER=$LC_ALL
export LC_NAME=$LC_ALL
export LC_ADDRESS=$LC_ALL
export LC_TELEPHONE=$LC_ALL
export LC_MEASUREMENT=$LC_ALL
export LC_IDENTIFICATION=$LC_ALL
EOF

If I used a quoted delimiter, $(SCRIPTPATH) and $(hostname) are no longer expanded. If I don't, I get the warning.

[tonistiigi]

I don't understand what you are describing as a bug in here. If you don't want vars evaluated then quotes are required. If you do want them evaluated then the warning is correct if you haven't defined LC_ALL.

[WanWizard]

The problem is that ${SCRIPTPATH} is a Dockerfile variable and SHOULD be evaluated, and $(hostname) is something that should run at runtime, and SHOULD NOT be evaluated.

So I need to fix a way to either prevent the second (escape the $ perhaps?), or find an alternative for this HEREDOC because what I want won't work.

[thaJeztah]

I'm not at my computer to verify, but It should mostly match what you'd get doing the same here doc in a shell; so, in this case either escape the vars (\$), or possibly a better alternative could be to set those env-vars in the image instead of depending on a script to export them, e.g.

FROM xxxxx
ARG SCRIPTPATH=/some/default

# bake the ARG value into the image
ENV SCRIPTPATH=$SCRIPTPATH

same for the other one.

And in that case use quoted here doc so that buildkit doesn't expand vars

[WanWizard]

I was trying to avoid that, I don't need SCRIPTPATH in my environment.

If I don't use a quoted HEREDOC:

• SCRIPTPATH is correctly added to the PATH
• THE hostname call in TITLE is not affected and appears as "${hostname)" in the file in the generated image
• but LC_ALL is interpreted by docker, resulting in

export LC_ALL=en_US.utf8
export LANG=
export LC_CTYPE=
export LC_NUMERIC=
export LC_TIME=
export LC_COLLATE=
...

which is NOT good.

If I do use a quoted HEREDOC

• SCRIPTPATH is not replaced
• LC_ALL is still a variable so with be interpreted at runtime
• TITLE is ok as expected.

So I guess there is no other option than use a quoted HEREDOC, and use the ENV workaround for SCRIPTPATH.

[tonistiigi]

You can escape the individual $ with \ that you don't want evaluated.

[WanWizard]

That was my question earlier, I think I prefer that instead of the ENV workaround.

Thanks again, everyone.