Data Engineer/ Data CuratorPosted: 2 months ago
Role: Data Engineer/ Data Curator
Location: Minneapolis, MN
Type of employment: Full Time
Extensive experience on developing on SPARK . Using either pyspark, sparkR, SparkLY, SCALA that are supported under SPARK.
Extensive experience in doing "ETL and data cleansing on very large data sets (big data) using this technology.
Excellent experience and understanding on SPARK framework and libraries.
SPARK ARCHITECTURE, RDD, DAG, Driver, SPARK context, SPARK Cluster, Master, Worker, Spark Transformations, Actions)
(AWS EMR framework that supports SPARK)
SPARK under AWS vs SPARK on other environment like Hadoop :