Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

How does only output only dataframe columns to csv?

Use case: golfing on the CLI in a utility function that I can’t afford to make complicated.

I need to peek at only the column names only of a large file in binary format, and not the column names plus, say, the first data row.

In my current implementation, I have to write the burdensome command to peek at the first row of large files:

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

my-tool peek -n 1 huge-file.parquet | head -n 1 | tr ',' '\n' | less

What I would like is to:

my-tool peek --cols huge-file.parquet | tr ',' '\n' | less

or

my-tool peek --cols -d '\n' huge-file.parquet | less

Without getting complicated in python. I currently use the following mechanism to generate the csv:

out = StringIO()
df.to_csv(out)
print(out.getvalue())

Is there a DataFrame-ish way to output just the columns to out via to_csv(...) or similarly simple technique?

>Solution :

Maybe something like this?

import pandas as pd
import numpy as np


if __name__ == "__main__":
    # some fake data for setup
    np.random.seed(1)
    df = pd.DataFrame(
        data=np.random.random(size=(5, 5)),
        columns=list("abcde")
    )

    out = df.columns.to_frame(name="columns")
    out.to_csv("file.csv", index=False)
    print(out)
  columns
a       a
b       b
c       c
d       d
e       e

csv output

Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading