I am creating a dataset by concatinating different other datasets in order to get a large one. This is in the purpose of training a generative AI models on it in the field of NLP
df = pd.concat([df, df_1, df_2, df_3])
Then I try to save it into csv file to use it when I need
df.to_csv('java_dataset.csv', index=False)
But I got this error
Error Traceback (most recent call last)
<ipython-input-67-9e8f66e43ade> in <cell line: 1>()
----> 1 df.to_csv('java_dataset.csv', index=False)
5 frames
/usr/local/lib/python3.10/dist-packages/pandas/io/formats/csvs.py in _save_chunk(self, start_i, end_i)
318
319 ix = self.data_index[slicer]._format_native_types(**self._number_format)
--> 320 libwriters.write_csv_rows(
321 data,
322 ix,
writers.pyx in pandas._libs.writers.write_csv_rows()
Error: need to escape, but no escapechar set
>Solution :
Try:
df.to_csv('java_dataset.csv',index=False, quoting=3, escapechar='\\')