Skip to main content

TO-678 - Sr. IT Data Engineer

Contract Type: Contract

Posted Date: April 16, 2026

ECCO Select is a talent acquisition and consulting company specializing in people, process and technology solutions. We provide the talent behind the technology enabling our clients to achieve their goals. For more information about ECCO Select, visit us at www.eccoselect.com.

Position Title:     Sr. IT Data Engineer

Location Information  

Washington, D.C.

Position Responsibilities:


As a Sr. IT Data Engineer, you will play a crucial role in supporting the planning and implementation of data policies, practices, and projects that serve to acquire, control, protect, and optimize enterprise data assets. You will be responsible for designing, developing, deploying, and maintaining robust data pipelines and processing systems, ensuring the delivery of clean, well-structured, and high-quality data for analytical and business needs.

Specific responsibilities include:

  • Design, build, and maintain data pipelines and processing solutions in Cloudera Data Platform environments.
  • Develop ETL/ELT workflows for ingesting data from diverse internal and external sources, ensuring alignment with data quality and format requirements.
  • Clean, transform, enrich, and aggregate structured and unstructured data into usable datasets and scalable databases.
  • Troubleshoot and resolve integration, content, and data flow issues across distributed systems.
  • Optimize the performance and reliability of data solutions, handling multiple data streams and high-volume workloads.
  • Implement code-based data engineering solutions primarily using Python, SQL, and tools such as PySpark, pandas, and dbt.
  • Develop and manage distributed data systems leveraging Apache NiFi, Hadoop, MapReduce, Hive, HBase, Kafka, and Spark.
  • Support source control, release management, and automatic deployment processes with Git and DevOps/CI/CD methodologies.
  • Build and sustain continuous integration and delivery pipelines for data platform operations.
  • Work within Agile environments, participating in Scrum and Kanban frameworks.
  • Participate in development and operational activities on UNIX/Linux platforms, including scripting and command-line tasks.
  • Collaborate with cross-functional teams, including developers, analysts, architects, and business stakeholders, to deliver effective and scalable data solutions.

 

Essential Skills, Experience


  • 5+ years of experience in application and/or data development, with extensive hands-on experience in Python.
  • 5+ years of experience with data integration and ingestion tools, particularly Apache NiFi.
  • Proven ability to develop, operate, and support long-running data pipelines within Cloudera Data Platform environments.
  • Advanced proficiency with SQL and Java for data manipulation and application development.
  • Experience working with Microsoft SQL Server.
  • Expertise with distributed data and computing environments, including: Apache NiFi, Hadoop, MapReduce, Hive, HBase, Kafka, and Spark.
  • Hands-on experience with PySpark, pandas, and/or dbt for complex data engineering tasks.
  • Strong background in Git-based source control and DevOps development practices.
  • Experience implementing and maintaining CI/CD pipelines in a data platform context.
  • Experience working in Agile development environments using Scrum and Kanban approaches.
  • Comfort working in UNIX/Linux operating systems, with knowledge of basic commands and shell scripting.
  • Excellent analytical, problem-solving, and troubleshooting skills.
  • Strong written and oral communication skills for effective team collaboration and stakeholder engagement.
  • U.S. Citizenship required.

Qualifications:

  • Bachelor's degree in Computer Science, Information Systems, Engineering, or a related technical field, or equivalent experience.
  • Experience supporting enterprise data environments in regulated or large-scale organizations is a plus.
  • Background with data quality controls, monitoring, and production pipeline operational support is desirable.
  • Knowledge of streaming and batch data architectures is preferred.
  • Experience with performance tuning and optimization in distributed data environments is an advantage.

ECCO Select is committed to hiring and retaining a diverse workforce. Our policy is to provide equal opportunity to all people without regard to race, color, religion, national origin, ancestry, marital status, veteran status, age, disability, pregnancy, genetic information, citizenship status, sex, sexual orientation, gender identity or any other legally protected category. Veterans of our United States Uniformed Services are specifically encouraged to apply for ECCO Select opportunities.

Equal Employment Opportunity is The Law
This Organization Participates in E-Verify