Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

Invalid comparison between dtype=datetime64[ns] and Timestamp

In Python I am trying to remove rows of a dataframe if the dates are lower than the dates of another dataframe. But the comparison does not work.

Here are my two dataframes and the results I try to compare. print(MeteoCH.head()) will result in:

                       TempAvg  TempMin  TempMax
Date                                                                      
2021-11-15 00:00:00      4.4      4.3      4.5     
2021-11-15 01:00:00      4.3      4.3      4.3     
2021-11-15 02:00:00      4.1      4.1      4.2     
2021-11-15 03:00:00      4.0      3.8      4.1    
2021-11-15 04:00:00      3.6      3.4      3.8    

And print(PicoLog.head()) will result in:

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

                           Temp1   Temp2   Temp3   
Date                                                                        
2021-11-15 18:34:18+01:00  21.268  21.671  21.190     
2021-11-15 18:34:20+01:00  21.266  21.673  21.194     
2021-11-15 18:34:22+01:00  21.270  21.680  21.194     
2021-11-15 18:34:24+01:00  21.263  21.673  21.180    
2021-11-15 18:34:26+01:00  21.262  21.672  21.185

If I try to execute the following command:

MeteoCH.drop(MeteoCH[MeteoCH.index < PicoLog.index.min()], inplace=True)

It results with the following error :

TypeError: Invalid comparison between dtype=datetime64[ns] and Timestamp

Why? How to solve it ?

I tried to "convert" it somehow, but it does not work.

Can someone help me please ?

>Solution :

Simplier is filter by greater or equal, inverted < like:

MeteoCH[MeteoCH.index >= PicoLog.index.min()]

MeteoCH[~(MeteoCH.index < PicoLog.index.min())]

Your solution is possible change filtering MeteoCH.index, but in my opinion overcomplicated:

MeteoCH.drop(MeteoCH.index[MeteoCH.index < PicoLog.index.min()], inplace=True)

EDIT:

Original problem was timezone offset, solution is DatetimeIndex.tz_localize:

PicoLog.index = PicoLog.index.tz_localize(None)
Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading