Lead Software Engineer (Data)
Company: The Judge Group
Posted on: May 27, 2023
Location: Philadelphia, PA Description: Our client is currently
seeking a Lead Big Data Developer This is a W2 Only Position and
must sit on one of the following locations
- Philadelphia, PA
- Charlotte, NC
- Dallas, TX
- Palo Alto, CA
- San Francisco, CA Must have recent experience with Pyspark &
Apache Airflow. Responsibilities: The Big Data Lead software
engineer is responsible for owning and driving the technical
innovation along with big data technologies. The individual is a
subject matter expert technologist with strong Python experience
and deep hands-on experience building data pipelines for the Hadoop
platform as well as Google cloud. This person will be part of
successful Big Data implementations for large data integration
initiatives. The candidates for this role must be willing to push
the limits of traditional development paradigms typically found in
a data-centric organization while embracing the opportunity to gain
subject matter expertise in the cyber security domain. Required
- 7+ years of Big Data Platform (data lake) and data warehouse
engineering experience demonstrated through prior work experiences.
Preferably with Hadoop stack: HDFS, Hive, SQL, Spark, Spark
Streaming, Spark SQL, HBase, Kafka, Sqoop, Atlas, Flink, Kafka,
Cloudera Manager, Airflow, Impala, Hive, HBase, Tez, Hue, and a
variety of source data connectors.
- 3+ years of hands-on experience building modern, resilient, and
secure data pipelines, including movement, collection, integration,
transformation of structured/unstructured data with built-in
automated data controls, and built-in logging/monitoring/alerting,
and pipeline orchestration managed to operational SLAs. Preferably
using Airflow, DAGS, connector plugins.
- 1+ years of experience with Google cloud data services such as
cloud storage, cloud proc, cloud flow, and Big Query.
- 5+ years of strong Python and other functional programming
skills Contact: This job and many more are available through The
Judge Group. Find us on the web at
Keywords: The Judge Group, Philadelphia , Lead Software Engineer (Data), IT / Software / Systems , Philadelphia, Pennsylvania
Didn't find what you're looking for? Search again!
Loading more jobs...