Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

Remove multiple file extesions when using gnu parallel and cat in bash

I have a csv file (separated by comma), which contains

file1a.extension.extension,file1b.extension.extension
file2a.extension.extension,file2b.extension.extension

Problem is, these files are name such as file.extension.extension

I’m trying to feed both columns to parallel and removing all extesions

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

I tried some variations of:

cat /home/filepairs.csv | sed 's/\..*//' | parallel --colsep ',' echo column 1 = {1}.extension.extension column 2 =  {2} 

Which I expected to output

column 1 = file1a.extension.extension column 2 = file1b
column 1 = file2a.extension.extension column 2 = file2b

But outputs:

column 1 = file1a.extension.extension column 2 = 
column 1 = file2a.extension.extension column 2 =

The sed command is working but is feeding only column 1 to parallel

>Solution :

As currently written the sed only prints one name per line:

$ sed 's/\..*//'  filepairs.csv
file1a
file2a

Where:

  • \. matches on first literal period (.)
  • .* matches rest of line (ie, everything after the first literal period to the end of the line)
  • // says to remove everything from the first literal period to the end of the line

I’m guessing what you really want is two names per line … one sed idea:

$ sed 's/\.[^,]*//g'   filepairs.csv
file1a,file1b
file2a,filepath2b

Where:

  • \. matches on first literal period (.)
  • [^,]* matches on everything up to a comma (or end of line)
  • //g says to remove the literal period, everything afterwards (up to a comma or end of line), and the g says to do it repeatedly (in this case the replacement occurs twice)

NOTE: I don’t have parallel on my system so unable to test that portion of OP’s code

Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading