I have searched everywhere but couldn’t quite find the exact similar solution to my problem. In Bash, I have a file that is tab delimited. It can potentially have several million lines. In the 27th column there is a string of colors which is delimited by a forward slash. My end goal is for the… Read More Taking Substring from Column to grab text until 3rd occurrence of '/'

I have a csv file and bash script but it does not work as needed The script #!/bin/bash #create path to redirect accounts.csv to same directory as accounts_new.csv path=$(dirname $1) awk ‘ BEGIN { FS="\""; OFS="," } # input is delimited by double qutoes NR==1 { print; next } { line="" for (i=1;i<NF;i+=2) { #… Read More Bash script Find Duplicate Values in CSV file

I’m writing a one-liner using awk to join strings within a loop. Basically, there’s an array of strings (I call it a), then I have another copy of that array (I call it b which is cloned using the clone function). I concatenate the string from array a (which is m) and b (which is… Read More Join string within loop using awk (involve replacing an array with another)

I am trying to argument processing on awk. It works fine if I have no body in it: PK@rhel8:~/tmp-> cat testARG.awk #!/usr/bin/awk -f BEGIN{ argc = ARGC ; CmdName = ARGV[0] ; FirstArg = ARGV[1] printf("Argument count = %d; command name = %s; first argument = %s\n",argc,CmdName,FirstArg) ; } #{ # printf("Argument count = %d;… Read More Cannot use Awk with arguments

I’ve got a file that looks like this: a 12345 b 3456 c 45678 and i’ve got bash array of strings: mylist=("a" "b") What I want to do is to sum numbers in second column but only for rows where first column value (aka "a" or "b") is present in mylist. My not-working code: cat… Read More How to check if string is present in bash array using awk

Given files test1 and test2: $ cat test* alert_name,id,severity bar,2,1 foo,1,0 alert_name,id,severity foo,1,9 bar,2,1 I want to find rows where name is the same but severity has changed (ie foo) and print the change. I have got this far using awk: awk ‘BEGIN{FS=","} FNR>1 && NR==FNR { a[$1]; next } ($1 in a) && ($3… Read More Print rows where one column is the same but another is different

I have this two columns file : ctg0F chr_1 ctg1F chr_2 ctg2F chr_3 ctg3F chr_4 ctg4F chr_5 ctg5F chr_6 ctg6F chr_4 ctg7F chr_7 ctg8F chr_8 The first column has different values. I’d like to add an index only for the repeated values in the second column. Here chr4 appears twice, and so : ctg0F chr_1… Read More Add index when same column value

I need to find the position of a certain word in a text file I tried to do something like this but it didn’t work (no any output): For example txt file contains: A B C Tarun A12 1 Man B6 2 Praveen M42 3 So I need to find a word ,,Tarun" position the… Read More How to find the position of specific word in a txt file using awk?

I’m trying to create a 2 column CSV, where the first column is a name, and the second column is the value. I am able to do this with a nested loop, but I’m positive that AWK can do this much faster DATA=’POLICY_NAME01,VM001,VM002,VM003 POLICY_NAME02,VM004 POLICY_NAME03,VM005,VM006′ IFS=$’\n’ for LINE in $DATA; do POLICY=$(echo "${LINE}" | cut… Read More Use awk to repeat first column in CSV with unknown amount of columns

I do have a file with different length of each line. E.g.: a; 1; 2; 3; 4; b; 11; 22; c; 122; 233; 344; 45; 56; d; 13; e; 144; 25; 36; 47; 58; 69; I try to generate a file, separated by semicolon where each line has the same amount of values. E.g.: a;… Read More awk adding a semicolon if empty