Lead Software Engineer (Data)

Company: The Judge Group
Location: Philadelphia
Posted on: May 27, 2023

Job Description:

Location: Philadelphia, PA Description: Our client is currently seeking a Lead Big Data Developer This is a W2 Only Position and must sit on one of the following locations

• Philadelphia, PA
• Charlotte, NC
• Dallas, TX
• Palo Alto, CA
• San Francisco, CA Must have recent experience with Pyspark & Apache Airflow. Responsibilities: The Big Data Lead software engineer is responsible for owning and driving the technical innovation along with big data technologies. The individual is a subject matter expert technologist with strong Python experience and deep hands-on experience building data pipelines for the Hadoop platform as well as Google cloud. This person will be part of successful Big Data implementations for large data integration initiatives. The candidates for this role must be willing to push the limits of traditional development paradigms typically found in a data-centric organization while embracing the opportunity to gain subject matter expertise in the cyber security domain. Required Qualifications:
  • 7+ years of Big Data Platform (data lake) and data warehouse engineering experience demonstrated through prior work experiences. Preferably with Hadoop stack: HDFS, Hive, SQL, Spark, Spark Streaming, Spark SQL, HBase, Kafka, Sqoop, Atlas, Flink, Kafka, Cloudera Manager, Airflow, Impala, Hive, HBase, Tez, Hue, and a variety of source data connectors.
  • 3+ years of hands-on experience building modern, resilient, and secure data pipelines, including movement, collection, integration, transformation of structured/unstructured data with built-in automated data controls, and built-in logging/monitoring/alerting, and pipeline orchestration managed to operational SLAs. Preferably using Airflow, DAGS, connector plugins.
  • 1+ years of experience with Google cloud data services such as cloud storage, cloud proc, cloud flow, and Big Query.
  • 5+ years of strong Python and other functional programming skills Contact: This job and many more are available through The Judge Group. Find us on the web at

Keywords: The Judge Group, Philadelphia , Lead Software Engineer (Data), IT / Software / Systems , Philadelphia, Pennsylvania