AWS Certified Data Engineer - Associate DEA-C01 — Topic 1 Question 222

Question

A company wants to build a dimension table in an Amazon S3 bucket. The bucket contains historical data that includes 10 million records. The historical data is 1 TB in size. A data engineer needs a solution to update changes for up to 10,000 records in the base table every day. Which solution will meet this requirement with the LOWEST runtime?

Accepted Answer

Correct answer: D. The correct answer is D. Option D: Develop an Amazon EMR job to read new changes into Apache Spark DataFrames. Use the Apache Hudi framework to create the base table in Amazon S3. Use the Spark update method to update the base table. Amazon S3 provides durable, scalable object storage that is fully managed. Amazon EMR runs big-data frameworks like Spark and Hadoop on managed clusters.

AWS Certified Data Engineer - Associate DEA-C01 — Question 222