I’m tryng to pass parameters into my query sql using parameters parameter into a RedshiftToS3Operator. I don’t understand how to use my parameters into my query. Here is my query into a mycode.sql file:
Select * From my_table where
date_start = {{ data_interval_end.subtract(days=day_as_parameters)}}
Using the Operator like this :
unload_data_to_s3 = RedshiftToS3Operator(
task_id="unload_data_to_s3",
s3_bucket=S3_BUCKET,
s3_key=S3_KEY,
select_query="mycode.sql",
unload_options=["CSV", "ALLOWOVERWRITE", "DELIMITER ','", "PARALLEL OFF", "HEADER"],
table_as_file_name=False,
parameters={"day_as_parameters": 10}
)
I tried several format but I can’t make it work.
>Solution :
You might have confused the parameters kwarg (for SQL templating) with params kwarg (for Jinja templating). Try using an airflow param instead of an SQL parameter:
Select * From my_table where
date_start = {{ data_interval_end.subtract(days=params.day_as_parameters)}}
unload_data_to_s3 = RedshiftToS3Operator(
task_id="unload_data_to_s3",
s3_bucket=S3_BUCKET,
s3_key=S3_KEY,
select_query="mycode.sql",
unload_options=["CSV", "ALLOWOVERWRITE", "DELIMITER ','", "PARALLEL OFF", "HEADER"],
table_as_file_name=False,
params={"day_as_parameters": 10}
)