Google Cloud Professional Data Engineer PDE — Topic 1 Question 177

Question

You want to rebuild your batch pipeline for structured data on Google Cloud. You are using PySpark to conduct data transformations at scale, but your pipelines are taking over twelve hours to run. To expedite development and pipeline run time, you want to use a serverless tool and SOL syntax. You have already moved your raw data into Cloud Storage. How should you build the pipeline on Google Cloud while meeting speed and processing requirements?

Accepted Answer

Correct answer: C. The correct answer is C. Option C: Ingest your data into BigQuery from Cloud Storage, convert your PySpark commands into BigQuery SQL queries to transform the data, and then write the transformations to a new table. Cloud Storage provides durable, scalable object storage that is fully managed. BigQuery is a serverless, petabyte-scale data warehouse for fast SQL analytics with no infrastructure to manage.

Google Cloud Professional Data Engineer PDE — Question 177