Nice To KnowIT Data Science & EngineeringComputing & ProcessingDataflow/Apache Beam
  • originally Google Cloud Dataflow (https://cloud.google.com/dataflow/)
  • now an Apache project (incubating) (http://incubator.apache.org/projects/beam.html)
  • comparison with Apache Spark 1.x: https://cloud.google.com/dataflow/blog/dataflow-beam-and-spark-comparison