Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

How to perform operation on csv without writing any files

I have a script that outputs and manipulates csv files. I want to do this without writing any files. Is there a way to hold everything in variables and get the same results? I am still new at this so I want to find a better way of doing things to keep my scripts clean. Any help with formatting or restructuring the script to make it more efficient would be very much appreciated.

# Export
$parsedbody1 | Export-Csv C:\windows\temp\parsedbody.Csv

# Import and select the desired properties
Import-Csv C:\windows\temp\parsedbody.Csv | Select-Object id, startepoch, ackedepoch | Export-csv C:\windows\temp\parsedbody1.csv

# Import and add column timetoack fill with 0
Import-Csv C:\windows\temp\parsedbody1.csv | Select-Object *,@{name='timetoack';Expression={'0'}} | Export-csv C:\windows\temp\parsedbody2.csv

# ackedepoch-startepoch=timetoack
$parsedbody2 = Import-Csv -path C:\windows\temp\parsedbody2.csv
  Foreach ($field in $parsedbody2) {
  $field.timetoack = $field.ackedepoch - $field.startepoch
  }
$parsedbody2 | Export-Csv -path C:\windows\temp\parsedbody3.csv

#remove all columns other than ID and timetoack
Import-Csv -path C:\windows\temp\parsedbody3.csv | Select * -ExcludeProperty startepoch | Export-Csv -path C:\windows\temp\parsedbody4.csv
Import-Csv -path C:\windows\temp\parsedbody4.csv | Select * -ExcludeProperty ackedepoch | Export-Csv -path C:\windows\temp\parsedbody5.csv

#Divide time to ack by 60 to get minutes
$parsedbody5 = Import-Csv -path C:\windows\temp\parsedbody5.csv
  Foreach ($field in $parsedbody5) {
  $field.timetoack = $field.timetoack/60
  }
$parsedbody5 | Export-Csv -path C:\windows\temp\parsedbody6.csv

#import final csv into a variable and cleanup files
$idtta = Import-Csv -path C:\windows\temp\parsedbody6.csv

del C:\windows\temp\parsedbody.csv
del C:\windows\temp\parsedbody1.csv
del C:\windows\temp\parsedbody2.csv
del C:\windows\temp\parsedbody3.csv
del C:\windows\temp\parsedbody4.csv
del C:\windows\temp\parsedbody5.csv
del C:\windows\temp\parsedbody6.csv

Here is what is contained in $parsedbody1 usually there are several of these contained in the variable:

id                            : DS2624511
type                          : dataSourceAlert
internalId                    : LMD513444
startEpoch                    : 1694574968
endEpoch                      : 0
acked                         : True
ackedEpoch                    : 1695400492
ackedBy                       : username@domain.com
ackComment                    : Comment
rule                          : Error
ruleId                        : 2
chain                         : Error
chainId                       : 5
subChainId                    : 0
nextRecipient                 : 113
receivedList                  : {"sms":["username"],"email":["username@domain.com"]}
severity                      : 3
cleared                       : False
sdted                         : False
SDT                           : 
alertValue                    : 113.5121
threshold                     : > 60 90 120
clearValue                    : 
monitorObjectId               : 2324
monitorObjectType             : device
monitorObjectName             : devicename - ipaddress
monitorObjectGroups           :       
resourceId                    : 24773
resourceTemplateId            : 420
resourceTemplateType          : DS
resourceTemplateName          : System Uptime
instanceId                    : 2914087
instanceName                  : WinSystemUptime
instanceDescription           : 
dataPointName                 : UptimeDays
dataPointId                   : 4136
detailMessage                 : 
customColumns                 : 
enableAnomalyAlertSuppression : 
enableAnomalyAlertGeneration  : 

>Solution :

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

Your code can be reduced to this using a calculated property with Select-Object:

$parsedbody1 |
    Select-Object id, @{ N = 'timetoack'; E = { $_.ackedepoch - $_.startepoch }} |
    Export-Csv 'theFinalThing.csv' -NoTypeInformation

If you don’t want to export the resulting array of objects, and hold them in memory instead:

$theFinalThing = $parsedbody1 |
    Select-Object id, @{ N = 'timetoack'; E = { $_.ackedepoch - $_.startepoch }}

If above syntax is hard to understand, you might have an easier time by just using PSCustomObjects:

$parsedbody1 | ForEach-Object {
    [pscustomobject]@{
        id        = $_.id
        timetoack = $_.ackedepoch - $_.startepoch
    }
} | Export-Csv ....
Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading