Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

Read CSV sheet in batches using Powershell

Recently I had written script which read data from csv and do some check, as csv data is huge i want to run it in batches so first 50 lines and execute them and write it to one folder and then execute next 50 lines and write output in another folder

below is the line i used to import and export csv file

$P = Import-Csv -Path .\Processes.csv

and export using

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

Export-Csv -Path "Data"

>Solution :

TotalCount is the best way to deal these scenarios. Instead of importing the csv, my recommendation would be to use Get-content and pick the necessary lines required:

Get-Content .\Processes.csv -TotalCount 50 | Out-File .\Processes_first50.csv

Another recommendation would be to use pipeline and then Select -First

Get-Content .\Processes.csv | select -First 50 | Out-File .\Processes_first50.csv

The last option is to use the -head parameter:

Get-Content .\Processes.csv -Head 50 > .\Processes_first50.csv

The > is the redirecting the output to a file which is similar to outfile but much more elegant.

Hope it helps.

Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading