Data Automation Engineer–REMOTE (Quarterly journey to DC) – Protos IT – Distant

Title: Data Automation Engineer
Location: REMOTE (Quarterly journey to DC)
period: 6 months, extension
safety clearance: means to efficiently get hold of a government-issued Public Trust clearance.

searching for a buyer expertise centered Data Automation Engineer to work with a workforce of material consultants and builders to design and implement modern knowledge automation options for Azure cloud-based knowledge lake, SQL, and NoSQL platform. As an information automation engineer, you’ll translate enterprise necessities to knowledge engineering and AI primarily based options to assist an enterprise scale Microsoft Azure primarily based knowledge analytics and reporting platform. Our excellent candidate is mission centered and supply oriented and applies important considering to create modern capabilities and clear up technical points.

In this function, you’ll:
Utilize Microsoft Azure providers together with Azure Data Factory, Synapse Pipelines, Apache Spark Notebooks, Python, SQL, saved procedures to develop excessive performing knowledge pipelines.
Continuously enhance and optimize the automation toolset for reliability, scalability, and flexibility.
Research and implement cutting-edge AI/ML and GenAI instruments to quickly develop clever purposes, scripts, and ETL pipelines that automate knowledge processes, and proactively get rid of workflow bottlenecks.
Work intently with implementation specialists, engineering groups, and buyer to know knowledge pushed wants and construct options that handle actual operational challenges.
Work intently with consumer personnel and workforce members to know knowledge necessities and develop acceptable knowledge options.
Identify, create, put together knowledge required for superior analytics, visualization, reporting, and AI/ML.
Implement knowledge migration, knowledge integrity, knowledge high quality, metadata administration, and knowledge safety capabilities to optimize knowledge pipelines.
Monitor and troubleshoot knowledge associated points to take care of excessive availability and efficiency.
Assist in ETL efficiency testing for knowledge pipelines by executing take a look at runs and validating knowledge load efficiency in opposition to anticipated benchmarks.
Support baseline efficiency evaluation by evaluating new pipeline efficiency with legacy system metrics (e.g., load time, throughput, latency).
Help monitor pipeline execution metrics corresponding to run time, knowledge quantity processed, and useful resource utilization to determine bottlenecks.
Participate in efficiency take a look at execution for Azure-based pipelines (ADF, Synapse, Databricks) in non-production environments.
Assist in figuring out efficiency points and contribute to tuning efforts (e.g., question optimization, partitioning, indexing fundamentals).
Validate knowledge consistency and completeness after efficiency take a look at runs to make sure no knowledge loss throughout high-volume processing.
Collaborate with DevOps and infrastructure groups to know how compute, reminiscence, and scaling influence pipeline efficiency.
Follow outlined efficiency testing processes, checklists, and pointers established by senior workforce members.
Document take a look at outcomes, points, and observations clearly for workforce assessment and monitoring.
Actively assist Agile DevOps course of, together with Program Increment planning.
Actively have interaction in steady studying to extend related expertise.
Maintain strict versioning and configuration management to make sure integrity of knowledge.

For this place, you must possess:
BS diploma in Computer Science or associated subject and a pair of+ years of expertise in related subject
2+ years of expertise with greater than one of many observe scripting languages: SQL, T-SQL, MDX/DAX, Python, and PySpark.
Experience designing and constructing ETL/knowledge engineering options, schedule and monitor using numerous cloud providers corresponding to Azure Data Lake Services, Azure Synapse Analytics, Azure Data Factory, Integration Runtime.
Experience working with Microsoft database and enterprise intelligence instruments, together with SQL Server, together with saved procedures, SSIS, SSRS, SSAS (cubes), and Power BI.
Experience with knowledge automation utilizing Azure/AWS CLI instruments with Bash or PowerShell scripting.
Familiarity with Azure DevOps Repos or GitHub and pipeline versioning/launch administration.
Demonstrated expertise in supporting manufacturing, testing, integration, and growth environments.
Open mindset, means to shortly adapt new applied sciences to unravel buyer issues.
Experience in Agile initiatives, working with a multi useful workforce.
Must be element oriented, and capable of assist a number of initiatives and duties.
Demonstrate steady studying to extend related expertise.

Not required, however further training, certifications, and/or expertise are a plus:
Experience and/or certifications in Generative AI growth, Generative AI for Data Analytics, and resolution supply.
Microsoft certification in Azure fundamentals, knowledge engineer, Power BI, AI or AWS licensed knowledge engineer.
Integration data utilizing enterprise/open supply ETL toolsets, REST API, Docker.
Experience or publicity to efficiency tremendous tuning (indexing, execution plans, views), knowledge profiling and question analytics.
Knowledge of safety compliance requirements with knowledge encryption, cloud digital networks, routing, firewalls, log analytics, screens
Basic data of ARM or Bicep templates for automation, familiarity with RBAC entry controls.
Data lineage and influence evaluation utilizing instruments like Purview, Synapse pipeline tracing, et