AWS Certified Data Engineer - Associate DEA-C01 — Topic 1 Question 223

Question

A data engineer develops an AWS Glue Apache Spark ETL job to perform transformations on a dataset. When the data engineer runs the job, the job returns an error that reads, “No space left on device.” The data engineer needs to identify the source of the error and provide a solution. Which combinations of steps will meet this requirement MOST cost-effectively? (Choose two.)

Accepted Answer

Correct answer: B, D. The correct answer is B, D. Option B: Use the Spark UI and AWS Glue metrics to monitor data skew in the Spark executors. Option D: Enable the --write-shuffie-files-to-s3 job parameter. Use the salting technique. Amazon S3 provides durable, scalable object storage that is fully managed. AWS Glue is a serverless ETL and data-catalog service that prepares and integrates data. This option delivers the requirement at the lowest cost.

AWS Certified Data Engineer - Associate DEA-C01 — Question 223