Executing Commands with Asterisks in Docker

Imagine you’re working on a Linux system, using Docker to manage your containers. You want to run a specific command within a running Docker container. The command you try is:

docker exec -t containername ls /tmp/sth/*

However, you encounter an error:

ls: cannot access '/tmp/bla/*': No such file or directory

Interestingly, when you access the container’s shell directly and run the same command, it works perfectly. What’s going wrong here?

The Root of the Problem

The issue arises from how shell globbing works in relation to Docker’s exec command. When you run the command from outside the container without invoking a shell inside the container, the glob pattern (/tmp/bla/*) is not expanded. Instead, it’s passed literally to the ls command inside the container. The Linux shell typically handles the expansion of these glob patterns, matching them to file names within the specified directory. However, since the outer shell cannot see the contents of the directory within the container, it fails to replace the glob pattern with the appropriate directory contents.

The Solution

To ensure the glob pattern is correctly expanded, you need to invoke a shell inside the container that can see its file system, which will handle the expansion as expected. Here’s how you can do it:

docker exec -t containername sh -c "ls /bla/sth/*"

Breakdown:

• docker exec: This command allows you to run specific commands within an already running container.

• -t: This flag allocates a pseudo-TTY, which simulates a real terminal, like ssh or an interactive shell session.

• containername: This is the name of your running Docker container.

• sh -c: The command sh invokes the shell within the container, and -c allows you to pass a string command to the shell. This sequence is crucial for handling complex commands or scripts.

• "ls /tmp/bla/*": Inside the quotes is the command you want the shell to execute. The quotes ensure that the glob pattern isn’t misunderstood or incorrectly split before the shell has a chance to interpret it.

Practical Example

Let’s say you are running a Ubuntu-based Docker container where you regularly fetch and accumulate temporary data in /tmp/bla. On inspecting this directory’s contents without physically logging into the container, the shell invocation method allows you to smoothly handle such operations, utilizing the powerful features of Linux shells seamlessly within Docker containers.

Conclusion

Understanding the interaction between Docker commands and Linux shell operations is fundamental when working in containerized environments. This knowledge allows for more efficient debugging and manipulation of containerized apps, ensuring that developers can maintain their flow without unnecessary interruptions. Remember, the power of Docker combined with Linux’s flexibility opens up a broad spectrum of possibilities for managing and deploying applications efficiently.

Related Reading

• Multiple Actions with a Single docker exec Call
• Copying Files Between Docker Containers and Host Machines
• How to Transfer docker Images Without a Repository
• Docker Compose useful commands
• Access Docker socket via TCP

What Else Breaks (and How to Fix It)

The sh -c pattern solves globbing, but it’s a trapdoor into a whole category of shell-quoting pain. Here are the other things that’ll bite you once you start leaning on docker exec for real automation work.

Quotes inside quotes. The moment your path or argument contains a space or a single quote, your carefully constructed one-liner falls apart. The safest escape hatch is to use bash instead of sh and lean on $'...' quoting or printf:

# path with a space — double-quoting the outer string keeps it intact
docker exec mycontainer sh -c 'ls "/var/log/my app/"'

# argument that itself contains a single quote
docker exec mycontainer sh -c 'grep '"'"'it'"'"'s broken'"'"' /var/log/app.log'
# ... yeah, don't do that. Use bash -c and $'...' instead:
docker exec mycontainer bash -c $'grep \'it\'s broken\' /var/log/app.log'

Honestly, if you’re wrangling quotes three layers deep, write a tiny script, copy it into the container with docker cp, and run it. Future-you deserves better than archaeology.

Minimal images don’t have bash. Alpine-based images ship with sh (busybox ash), not bash. If you hardcode bash -c and your container is Alpine, you’ll get:

OCI runtime exec failed: exec: "bash": executable file not found in $PATH

Check what shell exists first:

docker exec mycontainer which sh bash ash 2>/dev/null

Use whichever one responds. sh -c works on everything.

Pipes and redirects. Same problem as globbing — | and > are shell metacharacters, so they have to live inside the sh -c "..." string, not outside it:

# WRONG — the pipe is interpreted by your host shell
docker exec mycontainer cat /var/log/app.log | grep ERROR

# RIGHT — the whole pipeline lives inside the container shell
docker exec mycontainer sh -c 'cat /var/log/app.log | grep ERROR'

# Also RIGHT, and more efficient — grep reads the file directly
docker exec mycontainer sh -c 'grep ERROR /var/log/app.log'

Environment variables. If you need the container’s own env vars expanded (not your host’s), they also need to be inside the string:

# This expands $HOME on your HOST, not inside the container
docker exec mycontainer ls $HOME

# This expands $HOME inside the container — probably what you wanted
docker exec mycontainer sh -c 'ls $HOME'

The pattern is consistent: anything that needs to be evaluated by the container’s shell goes inside the quoted string. Anything you want evaluated by your host shell stays outside. Draw that line clearly and docker exec stops being a mystery.