Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

Parsing json output with JQ – Searching for patterns

I have the following json (based on ndpi output)

{"src_ip":"x.x.x.x","dest_ip":"x1.x1.x1.x1","src_port":48318,"dst_port":443,"ip":4,"proto":"TCP","ndpi": {"flow_risk": {"35": {"risk":"Susp Entropy","severity":"Medium","risk_score": {"total":210,"client":165,"server":45}}},"confidence": {"1":"Match by port"},"proto":"TLS","proto_id":"91","proto_by_ip":"Unknown","proto_by_ip_id":0,"encrypted":1,"breed":"Safe","category_id":5,"category":"Web"},"detection_completed":1,"check_extra_packets":0,"flow_id":0,"first_seen":1733074272.824,"last_seen":1733074282.757,"duration":9.933,"vlan_id":0,"bidirectional":1,"xfer": {"data_ratio":-0.958,"data_ratio_str":"Download","src2dst_packets":1268,"src2dst_bytes":86239,"src2dst_goodput_bytes":2551,"dst2src_packets":2693,"dst2src_bytes":4042956,"dst2src_goodput_bytes":3865218},"iat": {"flow_min":1,"flow_avg":6.1,"flow_max":4005,"flow_stddev":108.3,"c_to_s_min":0,"c_to_s_avg":7.4,"c_to_s_max":4005,"c_to_s_stddev":137.8,"s_to_c_min":0,"s_to_c_avg":1.4,"s_to_c_max":35,"s_to_c_stddev":2.2},"pktlen": {"c_to_s_min":66,"c_to_s_avg":68.0,"c_to_s_max":514,"c_to_s_stddev":28.8,"s_to_c_min":66,"s_to_c_avg":1501.3,"s_to_c_max":1506,"s_to_c_stddev":76.6},"tcp_flags": {"cwr_count":0,"ece_count":0,"urg_count":0,"ack_count":3961,"psh_count":1323,"rst_count":0,"syn_count":0,"fin_count":0,"src2dst_cwr_count":0,"src2dst_ece_count":0,"src2dst_urg_count":0,"src2dst_ack_count":1268,"src2dst_psh_count":7,"src2dst_rst_count":0,"src2dst_syn_count":0,"src2dst_fin_count":0,"dst2src_cwr_count":0,"dst2src_ece_count":0,"dst2src_urg_count":0,"dst2src_ack_count":2693,"dst2src_psh_count":1316,"dst2src_rst_count":0,"dst2src_syn_count":0,"dst2src_fin_count":0},"c_to_s_init_win":0,"s_to_c_init_win":0}

I can get almost all the information (src_ip, ports, etc), but there is one thing related to "flow_risk"
{"flow_risk": {"35": {"risk":"Susp Entropy","severity":"Medium"

related to the number (in this case 35). This number can be different (from 01 to 50) and I don’t know how to filter to search for different numbers.

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

Right now, this is the filter I’m using

cat data.json | jq -r '"\(.src_ip),\(.src_port),\(.dest_ip),\(.dst_port),\(.proto),\(.ndpi.proto),\(.ndpi.category),\(.ndpi.hostname),\(.duration),\(.vlan_id),\(.xfer.src2dst_bytes),\(.xfer.dst2src_bytes),\(.ndpi.flow_risk."35".risk),\(.ndpi.flow_risk."35".severity)"')

This works when the number is exactly 35, but is there any chanve to use something like wildcards with jq ? I mean how can I do this to accept multiple numbers ? (for ex: searching from 01 to 50 ?)

Something like
(.ndpi.flow_risk."".risk),(.ndpi.flow_risk."".severity)"’)

thanks !

>Solution :

To access a variable but known key, use the --arg flag to bind a variable to a value, and then .[$varname] to access the field:

jq -r --arg n "35" '… .ndpi.flow_risk[$n].risk …'

is there any chanve to use something like wildcards with jq ?

To access all keys, just use .[]. However, you presumably want to reference the same keys both times you’ve used 35 in the example. For that, iterate before creating (the relevant part of) the output string, and reference the iteration value (context):

jq -r '… ,\(.ndpi.flow_risk[] | "\(.risk),\(.severity)")'

To access a specific range, use range to create them, turn it into a string using tostring, and proceed as above ("0" + prepends the zero, and [-2:] reduces it to the last two characters):

jq -r '… ,\(.ndpi.flow_risk[range (50)+1 | ("0" + tostring)[-2:]] | "\(.risk),\(.severity)")'
Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading