Recursively walk a directory hierarchy with ‘bash’ globstar **

2 minute read  

How to list all .js files in nested folders? We can use the ** glob pattern.

Assuming we have this:

├── d1
│   ├── d11
│   │   ├── d111
│   │   │   └── f5.js
│   │   └── f4.js
│   ├── d12
│   ├── f2.js
│   └── f3.js
├── d2
│   └── f6.js
└── f1.js
5 directories, 6 files

By default ** behaves exactly like * for file names and directory names:

$ ls *.js
f1.js

$ ls **.js
f1.js
# */ only matches first-level directories
$ ls */*.js
d1/f2.js  d1/f3.js  d2/f6.js

# Similarly, **/ only matches first-level directories
$ ls **/*.js
d1/f2.js  d1/f3.js  d2/f6.js

# Same behavior with **/
$ ls **/**.js
d1/f2.js  d1/f3.js  d2/f6.js

This isn’t quite what we want. Instead of matching only the first-level, we want ** to match nested directories recursively so that we can list all 6 files in the above example.

Set globstar option

GNU bash v4+ manual says that we can set the globstar option to make ** recursively match nested directories.

globstar

If set, the pattern ‘**’ used in a filename expansion context will match
all files and zero or more directories and subdirectories. If the pattern
is followed by a ‘/’, only directories and subdirectories match.

Use the shell optional behavior command shopt to do so:

# Before
$ shopt | grep globstar
26:globstar       	off

# Set the option
$ shopt -s globstar

# After
$ shopt | grep globstar
26:globstar       	on

If you run into an error -bash: shopt: globstar: invalid shell option name on Mac, you need to update Mac terminal to use bash v4+.

Demo folder for illustration

# Make a demo folder
$ mkdir demo-globstar
$ cd demo-globstar/

# Make 2 main folders, one with nested sub-folders inside
$ mkdir -p d1/d11/d111 d1/d12 d2

# Create js files
$ touch f1.js d1/f2.js d1/f3.js d1/d11/f4.js d1/d11/d111/f5.js d2/f6.js

# We'll then have this
demo-globstar/
├── d1
│   ├── d11
│   │   ├── d111
│   │   │   └── f5.js
│   │   └── f4.js
│   ├── d12
│   ├── f2.js
│   └── f3.js
├── d2
│   └── f6.js
└── f1.js
5 directories, 6 files

** should now match directory names recursively

# We should now get all 6 files
$ ls **/*.js
d1/d11/d111/f5.js  d1/f2.js           d2/f6.js
d1/d11/f4.js       d1/f3.js           f1.js

We can use ** for more meaningful purposes like copying files:

# Copy all .js files under d1/ to d2/
$ cp d1/**/*.js d2/

# f5, f4, f2, f3.js files should have been copied to d2/
demo-globstar/
├── d1
│   ├── d11
│   │   ├── d111
│   │   │   └── f5.js
│   │   └── f4.js
│   ├── d12
│   ├── f2.js
│   └── f3.js
├── d2
│   ├── f2.js
│   ├── f3.js
│   ├── f4.js
│   ├── f5.js
│   └── f6.js
└── f1.js
5 directories, 10 files

Leave a Comment