I have a dataframe df with a date column of strings like this one :
Date
01/06/2022
03/07/2022
18/05/2022
12/02/2021
WK28
WK30
15/09/2021
09/02/2021
...
I want to update my dataframe with the last 6 months data AND the wrong format data (WK28, WK30…) like this :
Date
01/06/2022
03/07/2022
18/05/2022
WK28
WK30
...
I managed to keep the last 6 months dates by converting the column to Date format and computing a mask with a condition :
df['Dates']=pd.to_datetime(df['Dates'], errors='coerce', dayfirst=True)
mask = df['Dates'] >= pd.Timestamp((datetime.today() - timedelta(days=180)).date())
df = df[mask]
But how can I also keep the wrong format data ?
>Solution :
Use boolean indexing with 2 masks:
# save date as datetime in series
date = pd.to_datetime(df['Date'], errors='coerce', dayfirst=True)
# is it NaT?
m1 = date.isna()
# is it in the last 6 months?
m2 = date.ge(pd.to_datetime('today')-pd.DateOffset(months=6))
# if any condition is True, keep the row
out = df[m1|m2]
output:
Date
0 01/06/2022
1 03/07/2022
2 18/05/2022
4 WK28
5 WK30
intermediate masks:
Date m1 m2 m1|m2
0 01/06/2022 False True True
1 03/07/2022 False True True
2 18/05/2022 False True True
3 12/02/2021 False False False
4 WK28 True False True
5 WK30 True False True
6 15/09/2021 False False False
7 09/02/2021 False False False