A public sector organization ingests large datasets from various relational databases into an Amazon S3 data lake on a daily basis. Data analysts need a mechanism to profile the data and diagnose data quality issues after the data is ingested into Amazon S3. The solution should allow the data analysts to visualize and explore the data quality metrics through a user interface.
Which set of steps provide a solution that meets these requirements?
A. Create a new AWS Glue DataBrew dataset for each dataset in the S3 data lake. Create a new DataBrew project for each dataset. Create a profile job for each project and schedule it to run daily. Instruct the data analysts to explore the data quality metrics by using the DataBrew console.
B. Create a new AWS Glue ETL job that uses the Deequ Spark library for data validation and schedule the ETL job to run daily. Store the output of the ETL job within an S3 bucket. Instruct the data analysts to query and visualize the data quality metrics by using the Amazon Athena console.
C. Schedule an AWS Lambda function to run daily by using Amazon EventBridge (Amazon CloudWatch Events). Configure the Lambda function to test the data quality of each object and store the results in an S3 bucket. Create an Amazon QuickSight dashboard to query and visualize the results. Instruct the data analysts to explore the data quality metrics using QuickSight.
D. Schedule an AWS Step Functions workflow to run daily by using Amazon EventBridge (Amazon CloudWatch Events). Configure the steps by using AWS Lambda functions to perform the data quality checks and update the catalog tags in the AWS Glue Data Catalog with the results. Instruct the data analysts to explore the data quality metrics using the Data Catalog console.
Which set of steps provide a solution that meets these requirements?
A. Create a new AWS Glue DataBrew dataset for each dataset in the S3 data lake. Create a new DataBrew project for each dataset. Create a profile job for each project and schedule it to run daily. Instruct the data analysts to explore the data quality metrics by using the DataBrew console.
B. Create a new AWS Glue ETL job that uses the Deequ Spark library for data validation and schedule the ETL job to run daily. Store the output of the ETL job within an S3 bucket. Instruct the data analysts to query and visualize the data quality metrics by using the Amazon Athena console.
C. Schedule an AWS Lambda function to run daily by using Amazon EventBridge (Amazon CloudWatch Events). Configure the Lambda function to test the data quality of each object and store the results in an S3 bucket. Create an Amazon QuickSight dashboard to query and visualize the results. Instruct the data analysts to explore the data quality metrics using QuickSight.
D. Schedule an AWS Step Functions workflow to run daily by using Amazon EventBridge (Amazon CloudWatch Events). Configure the steps by using AWS Lambda functions to perform the data quality checks and update the catalog tags in the AWS Glue Data Catalog with the results. Instruct the data analysts to explore the data quality metrics using the Data Catalog console.