Cloud

Google Dataflow Interview Questions

4.4 (342)12,342 learners21 lessons41m

Curriculum

Topic

01 - What is Google Cloud Dataflow, and how does it relate to Apache Beam1:53
02 - What does it mean that Apache Beam is a unified model for batch and streaming, and why is that valuable1:56
03 - What is a PCollection, and what is the difference between a bounded and an unbounded PCollection2:00
04 - What is a PTransform, and what is the difference between ParDo, Map, and FlatMap2:05
05 - What is a DoFn, and how does ParDo use it to process elements in parallel2:02
06 - What is a Pipeline in Beam, and why is it described as a directed acyclic graph (DAG)2:03
07 - What are IO connectors (sources and sinks), and name a few commonly used with Dataflow1:58
08 - What is the difference between event time and processing time, and why does it matter in streaming1:49
09 - What is windowing, and what are the main window types (fixedtumbling, slidinghopping, session)2:01
10 - What is a watermark in Dataflow, and how does it relate to late data1:45
11 - What are triggers, and how do allowed lateness and accumulation mode affect windowed results1:52
12 - Why can't you group an unbounded PCollection by key alone, and how do windows solve this1:53
01 - What is the difference between GroupByKey, CoGroupByKey, and Combine (CombinePerKey)1:40
02 - You need to read events from PubSub, aggregate them into 1-minute counts, and write the results to BigQuery. W1:58
03 - Your ParDo needs to enrich each record with reference data (e.g., a lookup table). How would you make that ref2:01
04 - A single transform needs to split records into valid and invalid outputs and route them differently. How would1:55
05 - You need to join a high-volume event stream with a slowly changing dataset where the matching record may arriv1:50
06 - You want the same pipeline code to run as a nightly batch job and as a continuous streaming job. How does Beam1:53
07 - A streaming pipeline can't keep up with incoming data and the backlog (system lag) keeps growing. How would yo2:09
08 - One step in your pipeline is a bottleneck because most records share the same key (hot key data skew). How wou1:52
09 - Your Dataflow job costs are higher than expected. What levers (autoscaling, Streaming Engine, Dataflow Prime,1:59