How to parse each line separately with awk?

March 26, 2023

A multi-line variable LOG_BUF is set in a bash script parse.sh. Then the variable is parsed with awk, printing all rows containing pat:

#!/bin/bash

LOG_BUF=$(cat <<-END
    pat TEST_a
    pat TEST_b
    TEST_c
    pat TEST_d
END
)
echo ${LOG_BUF} | awk 'BEGIN{}; /pat/{printf("%d %s", NR, $0); printf("\n")}; END{printf("\n")}'

The expected output is:

$ ./parse.sh
1 mem TEST_a
2 mem TEST_b
4 mem TEST_d

But instead, it prints:

$ ./parse.sh
1 mem TEST_a mem TEST_b TEST_c mem TEST_d

Seemingly, awk treats the whole string as a single record. How to get awk to parse the string as a multi-line string?

>Solution :

Use double quotes to preserve the newlines

You must sandwich the LOG_BUF between double quotes to preserve the newlines, so change the last line to this:

$ echo "${LOG_BUF}" | awk 'BEGIN{}; /pat/{printf("%d %s", NR, $0); printf("\n")}; END{printf("\n")}'
1 pat TEST_a
2     pat TEST_b
4     pat TEST_d

Good to know

On another note, you may want to use read instead of cat to assign a heredoc value to a variable in bash, the following code snippet is inspired from this SO answer, I encourage you to check it out for more details:

$ read -r -d '' LOG_BUF <<-'EOF'
    pat TEST_a
    pat TEST_b
    TEST_c
    pat TEST_d
EOF

$ echo "$LOG_BUF"
pat TEST_a
    pat TEST_b
    TEST_c
    pat TEST_d