Copy and Rename files in S3 bucket Lambda

April 6, 2023

I am writing a lambda function to test the tables in Athena and I have gotten that part to work. I am trying to rename the ResultConfiguration outputlocation file name that the function creates which cannot be renamed unless it’s copied and deleted. I am getting Invalid Bucket Name everytime it tries to copy and I’m not sure why. This is the python code I have after doing some research:

def copy_delete_output_results(query_id, table_name):
    oldCsvFileLocation = output + query_id + '.csv'
    newOutputLocation = output + table_name
    newFileName = table_name + '.csv'
    s3.Object(newOutputLocation,newFileName).copy_from(CopySource=oldCsvFileLocation)
    s3.Object(output,oldCsvFileLocation).delete()



oldCsvFileLocation = s3://ab-binaries-bucket-devtest/test/testQueryResults/1d8ab6ce-eda0-435f-93c3-2baa6fec8abd.csv
newOutputLocation = s3://ab-binaries-bucket-devtest/test/testQueryResults/dsm_abc_dns
newFileName = dsm_abc_dns.csv

Error Message:

Invalid bucket name "s3://ab-binaries-bucket-devtest/test/testQueryResults/dsm_abc_dns": Bucket name must match the regex "^[a-zA-Z0-9.\-_]{1,255}$" or be an ARN matching the regex "^arn:(aws).*:(s3|s3-object-lambda):[a-z\-0-9]*:[0-9]{12}:accesspoint[/:][a-zA-Z0-9\-.]{1,63}$|^arn:(aws).*:s3-outposts:[a-z\-0-9]+:[0-9]{12}:outpost[/:][a-zA-Z0-9\-]{1,63}[/:]accesspoint[/:][a-zA-Z0-9\-]{1,63}$"

>Solution :

s3.Object has two parameteres, the first one is the bucket name (string), and the second one is the file key (string).
For example:

import boto3

s3 = boto3.resource('s3')

source_bucket_name = 'ab-binaries-bucket-devtest'
source_file_key = 'test/testQueryResults/1d8ab6ce-eda0-435f-93c3-2baa6fec8abd.csv'

destination_bucket_name = 'ab-binaries-bucket-devtest'
destination_file_key = 'test/testQueryResults/dsm_abc_dns/dsm_abc_dns.csv'

# Copy the file
s3.Object(destination_bucket_name, destination_file_key).copy_from(
    CopySource={'Bucket': source_bucket_name, 'Key': source_file_key}
)
# Delete the original file
s3.Object(source_bucket_name, source_file_key).delete()

So, the error you got is because you are passing the full s3 path in the first param to the s3.Object instead of passing the bucket-name