I have trouble scheduling an AWS Batch Job

I am trying to schedule an AWS Batch Job using the AWS EventBridge. I try to create a schedule but then it gives me an error Invalid RequestJson provided. Reason Request payload is missing the following field(s): JobName, JobQueue, JobDefinition.

I selected AWS Batch as a target API when creating a schedule and in it, I also chose the AWS Batch Submit Job option.

This is the JSON input I wrote in the same Select Target Section in the Submit Job JSON input field.

{
  "jobName": "insiderTradersJob556",
  "jobDefinition": "arn:aws:batch:us-east-2:157733351594:job-definition/insiderTraders:1",
  "jobQueue": "arn:aws:batch:us-east-2:157733351594:job-queue/insider-traders-script",
  "dependsOn": [],
  "arrayProperties": {},
  "retryStrategy": {},
  "timeout": {},
  "parameters": {},
  "containerOverrides": {
    "command": [
      "python",
      "insiderTradersUploader.py"
    ],
    "resourceRequirements": [],
    "environment": []
  }
}

When submitting a job through AWS Batch using this same JSON input, it works perfectly fine. The job succeeded and it worked as expected.

My execution role is Amazon_EventBridge_Scheduler_ECS_4347190648

I tried using

{
  "jobName": "insiderTradersJob556",
  "jobDefinition": "arn:aws:batch:us-east-2:157733351594:job-definition/insiderTraders:1",
  "jobQueue": "arn:aws:batch:us-east-2:157733351594:job-queue/insider-traders-script"
}

but it still didn’t work.

I also tried using the AWS lambda functions. I tried creating a function with the container image i used in the AWS Batch (the image is saved in the Elastic Container Registry).However, it just said the image is not found 🙁

My execution role has multiple permissions. First one is:

    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": [
                    "batch:SubmitJob",
                    "batch:DescribeJobQueues"
                ],
                "Resource": "arn:aws:batch:us-east-2:157733351594:job-queue/insider-traders-script"
            }
        ]
    }

My second one is

    {
        "Statement": [
            {
                "Action": [
                    "batch:SubmitJob"
                ],
                "Effect": "Allow",
                "Resource": [
                    "arn:aws:batch:us-east-2:157733351594:job-definition/getting-started-wizard-job-definition:1",
                    "arn:aws:batch:us-east-2:157733351594:job-queue/insider-traders-script"
                ]
            }
        ],
        "Version": "2012-10-17"
    }

My other ones are AmazonRDSFullAccess, AWSBatchServiceRole and AmazonECSTaskExecutionRolePolicy. I am suspecting I need some other role.

>Solution :

It seems that you are encountering issues when scheduling an AWS Batch job using AWS EventBridge. Let’s go through the troubleshooting steps to help you resolve the problem.

First, make sure you have the necessary permissions to schedule a job on AWS Batch and publish events to AWS EventBridge. Verify that the execution role (Amazon_EventBridge_Scheduler_ECS_4347190648) associated with the event rule has the required permissions to submit jobs to the specified job queue in AWS Batch.

Next, double-check the JSON input you provided when creating the schedule. The error message indicates that the request payload is missing the following fields: JobName, JobQueue, and JobDefinition. Ensure that you have included these fields in the JSON input. Here’s an example:

{
  "detail": {
    "jobName": "insiderTradersJob556",
    "jobDefinition": "arn:aws:batch:us-east-2:157733351594:job-definition/insiderTraders:1",
    "jobQueue": "arn:aws:batch:us-east-2:157733351594:job-queue/insider-traders-script"
  }
}

Replace the detail field with your desired event pattern if needed.

If the issue persists, try creating a new AWS EventBridge rule and specify the target as "AWS Batch Submit Job." Provide the JSON input again, ensuring that all required fields are present.

Regarding your attempt to use AWS Lambda functions with the container image, it’s important to note that AWS Lambda currently supports running functions based on AWS Lambda layers and custom runtime APIs. It doesn’t directly support running container images from Elastic Container Registry (ECR). If you want to run a container image, you can consider using AWS Fargate or AWS ECS.

I hope these troubleshooting steps help you resolve the issue with scheduling your AWS Batch job. If you continue to face problems, providing more details or error messages you encounter can be helpful in further assisting you.