use regex and grep to get all unique instances of pattern in a log file

March 1, 2022

I need to get a list of unique client computer names/ip addresses that are accessing a server from the access logs of the server.

The target log line looks like this:

2020-11-17 15:34:04.208 -0500 Information 94  XYZ-ASDF-FMP123  Client "%USERNAME% (QWER-L1212-W6) [11.22.333.44]" opening database "databasename" as "username".

In this example, the string (QWER-L1212-W6) [11.22.333.44] would be an example of a unique instance of a client computer/ip address.

So the result would be something like this:

(QWER-L1212-W6) [11.22.333.44]
(QWER-L1234-W7) [11.22.333.55]
etc...

I tried this without success:

grep --only-matching '\(.+\) \[.+\]' | sort --unique Access.log

the matching fails and the entire log line is returned.

>Solution :

Note you are using a POSIX BRE regex flavor since you do not pass -E/-r nor -P options to change the regex flavor from the default one. \(...\) defines a capturing group in POSIX BRE. There are more issues here though.

You need to use

grep -o '([^()]*) \[[^][]*]' Access.log | sort -u

Note the location of the input file argument to grep.

The ([^()]*) \[[^][]*] here is a POSIX BRE pattern that matches

( – a literal ( char (a \( is the start of a capturing group)
[^()]* – zero or more chars other than ( and )
) – a literal ) char (a \) is the end of a capturing group)
– a space
\[ – a [ char
[^][]* – zero or more chars other than [ and ]
] – a ] char.

See the online demo:

#!/bin/bash
s='2020-11-17 15:34:04.208 -0500 Information 94  XYZ-ASDF-FMP123  Client "%USERNAME% (QWER-L1212-W6) [11.22.333.44]" opening database "databasename" as "username".'
grep -o '([^()]*) \[[^][]*]' <<< "$s" | sort -u
# => (QWER-L1212-W6) [11.22.333.44]