Big Data Engineer Location: Irving, TX Job Type: Full-Time (Hybrid) Experience: 7+ Years F2F Interview Job Summary We are seeking an experienced Big Data Engineer with 7+ years of professional experience in designing, developing, and implementing scalable data solutions. The ideal candidate will have strong hands-on experience with Hadoop, Spark, PySpark, Kafka, Hive, SQL, and cloud-based data platforms. The candidate will be responsible for building high-performance data pipelines, processing large and complex datasets, and working closely with data scientists, analysts, application developers, and business stakeholders. Key Responsibilities • Design, develop, and maintain scalable Big Data pipelines and data processing solutions. • Develop batch and real-time data processing applications using Apache Spark/PySpark. • Build and optimize ETL/ELT pipelines for large-scale data processing. • Work with Hadoop, HDFS, Hive, Spark, Kafka, and other Big Data technologies. • Develop complex SQL queries and optimize data processing jobs for performance. • Design data ingestion and transformation processes from multiple structured and unstructured sources. • Develop real-time streaming pipelines using Kafka and Spark Streaming. • Perform data cleansing, transformation, validation, and quality checks. • Troubleshoot data pipeline failures and optimize jobs for scalability and reliability. • Work with cloud-based data platforms and storage services. • Collaborate with data architects, data scientists, analysts, and engineering teams. • Implement data security, governance, and access-control best practices. • Participate in code reviews, testing, deployment, and production support. • Contribute to Agile/Scrum development activities and technical documentation. Required Skills • 7+ years of experience in Big Data Engineering • Strong hands-on experience with Apache Spark and PySpark • Strong programming experience with Python and/or Scala • Strong SQL skills and experience with relational databases • Hands-on experience with Hadoop, HDFS, Hive, and MapReduce • Experience with Apache Kafka and real-time data processing • Strong experience building ETL/ELT data pipelines • Experience with data ingestion, transformation, and processing of large datasets • Knowledge of distributed computing and Big Data architecture • Experience with data warehouse/data lake concepts • Experience with Git and CI/CD • Strong troubleshooting and performance-tuning skills Cloud & Modern Data Technologies Experience with one or more of the following is highly preferred: • AWS: S3, EMR, Glue, Redshift, Lambda • Azure: ADLS, Databricks, Azure Data Factory, Synapse • GCP: BigQuery, Dataproc, Cloud Storage • Databricks • Snowflake Preferred Qualifications • Experience developing both batch and streaming data pipelines. • Experience with Apache Airflow or similar workflow orchestration tools. • Knowledge of data modeling and data warehousing concepts. • Experience with Docker/Kubernetes is a plus. • Understanding of data governance, security, and data quality frameworks. • Experience working in Agile/Scrum environments. • Strong communication and analytical problem-solving skills.