File1:
foo
bar
baz
File2:
bar.patch
grep -f file1 file2 > file1
Expected result, file1 contains bar.patch, instead it is empty.
How is grep processing the input file such that I cannot redirect to it?
>Solution :
Redirection > happens before the command is executed, hence the grep sees empty file and returns nothing. If you really want to write to the same file you can use the tee command eg:
grep -f file1 file2 | tee file1
Check the bash manual for details:
Before a command is executed, its input and output may be redirected
using a special notation interpreted by the shell. Redirection allows
commands’ file handles to be duplicated, opened, closed, made to refer
to different files, and can change the files the command reads from
and writes to. Redirection may also be used to modify file handles in
the current shell execution environment. The following redirection
operators may precede or appear anywhere within a simple command or
may follow a command. Redirections are processed in the order they
appear, from left to right.
It appears that grep is reading the file1 at the start and then it keeps the result in its memory. However, since this behavior is not documented it can potentially suddenly change. Hence it is prudent to buffer the output until the grep finishes, and only then write it to the file1. In order to do that you could use for example the sponge program (it is part of the moreutils).
Example:
grep -f file1 file2 | sponge | tee file1
Or just redirect to a temporary file and then mv it over the file1.
grep -f file1 file2 > file3
mv -f file3 file1