AWS Certified Data Engineer Associate DEA-C01 Practice Question

Amazon S3 Select lets applications retrieve only the required rows and columns from objects that are stored in S3. Calling S3 Select from inside the Lambda function streams a much smaller response, so the function uses far less memory and completes well within the 15-minute execution limit. Increasing the timeout is impossible because Lambda's maximum is 15 minutes. Simply allocating more memory may reduce execution time, but the entire 3-GB object would still need to be downloaded, so timeouts could persist and costs would rise. Breaking the file into smaller parts would require additional preprocessing outside the current workflow and S3 multipart upload completion does not create multiple objects that automatically generate individual events. Replacing the trigger with an Amazon Redshift COPY command abandons the requirement to keep the existing Lambda-based design.