Exploring Bash Process Substitution
Recently, while debugging some old bash scripts, my pair and I came across a command that looked like
yq eval "$KEY = \"$VAL\"" -- <(echo "$CURRENT_NOTES")
My pair hadn't previously had the chance to fully grok the
<( things ) construct, so I attempted to explain. This post is a more considered version of the conversation that followed.
If you run a command that contains the term
<( things ), then bash will do the following:
- create an imaginary file for us
- pretend that that file contains the output of the command
- pass that file as an argument to the command we're running (in our case, the
This is called Process Substitution
You can actually see some of the nuts and bolts of process substitution in action if you use another cool bash feature:
A quick detour –
If you're not familiar with
set -x, this is a command which makes bash print out everything it does until you tell it to stop. So, you could have a script that looks like:
cd /tmp mkdir my-new-directory cd my-new-directory ls -a
If you run this script, all you will see output is this:
This is the output of your
ls -a command, in the empty directory you just created.
However, if you add
set -x at the beginning of the script, the output changes to:
+ cd /tmp + mkdir my-new-directory + cd my-new-directory + ls -a . ..
Each line beginning with a
+ is bash printing the command it's about to run, before it runs it. You can tell bash to stop doing this by using
set +x (note the + instead of the -).
To read more about this (and all the other cool things you can do with the
set command) you can do
info "(bash) The Set Builtin" on any mac or linux box
set -x to watch Process Substitution happening
Ok, so we want to understand process substitution, which looks like
<( things ) , using the tool
set -x. Let's write a tiny script:
set -x cat <(echo "Badgers!")
If you run that script, you'll see:
+ cat /dev/fd/63 ++ echo 'Badgers!' Badgers!
Let's break that down
The first thing that bash does is run
But what is this
Well, I think it's like a temporary file (but see ⚠ below). So our
cat command is going to read whatever is in that file, and print it to the terminal.
The next thing that bash does is
Notice that my double-quoted string has become a single-quoted one. That's because bash has already done all the variable substitution it's going to do, so this is exactly the command that's actually going to run. Also notice that this line begins with
++ not just
+. This is because bash needs to finish running this line before it can complete the last one that begin with a single
The result of that command goes "into"
/dev/fd/63 ready for cat to pick it up.
cat does its job, and the string
Badgers! appears on the terminal. And the script exits.
⚠ I've skirted close to implying that
/dev/fd/63 is a regular file, like a temporary file. It's not – or at least not always. Often it will be a thing called a "named pipe" or a "FIFO". But that's a topic for another post.
You can read more about process substitution by typing
info "(bash) Process Substitution" on any mac or linux box. There's also lots more cool bash documentation in
info bash. If using the
info system feels weird, you can do
info info to learn about it.