Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

why does bash 'read' expand a wildcard appearing in stdin to yield a directory listing?

I’m writing a script to process a data file, and started with these two command lines:

proc(){ echo $1 ; }

while read -r line ; do proc "$line";  done <data.csv | less

To my surprise, I found a listing of the current directory in the middle of the output, and realised it appears where the data file contains an asterisk character.

The expansion occurs before proc is called. Could someone please explain why this is happening. Is it happening in the command line processor or read or the file stream?

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

I can do this:

cat data.csv | tr '*' '_' | while read -r line ; do proc "$line";  done | less

but I’d like to know if there’s a way to prevent this expansion.

Alternatively, is there a way to filter input using the redirection form, rather than the pipe, and what other special characters, eg: [,],?,- would I need to replace?

Cheers,
bitrat

PS: An example data file:

gfh fg hfgh fgh fg hgh 
7 4 674 547767 56 7 56756
ghdghh gh fg h fgh fg hf gh
8 678 678 * 67 867 8 678 
gfh fg hfgh fgh fg hgh 
7 4 674 547767 56 7 56756
ghdghh gh fg h fgh fg hf gh
8 678 678 67 867 8 678 

And example output:

gfh fg hfgh fgh fg hgh
7 4 674 547767 56 7 56756
ghdghh gh fg h fgh fg hf gh
8 678 678 data.csv file1.txt file2.txt file3.txt  67 867 8 678
gfh fg hfgh fgh fg hgh
7 4 674 547767 56 7 56756
ghdghh gh fg h fgh fg hf gh
8 678 678 67 867 8 678

UPDATE:

The asterisk is only treated as a wildcard if it has spaces on either side, so must be getting tokenised at some point.

UPDATE2:

The expansion doesn’t occur if echo is called directly..

while read -r line ; do echo "$line";  done <data.csv

>Solution :

Why: Pathname expansion happens later than variable expansion. Search for "order of expansions" in man bash.

The expansion does not occur before proc is called, it happens inside proc.

Solution: Add double quotes:

echo "$1"
Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading