Senior Azure Data Engineer Job at Programmers.io, Cary, NC

RExnSTVFbkxEeitxZG40WXVST1BvWnBrWUE9PQ==
  • Programmers.io
  • Cary, NC

Job Description

Local to Cary NC only!


Design and build batch and streaming data pipelines using Azure, Databricks, and Kafka.
Develop ETL/ELT solutions using Spark, SQL and Python/Pyspark, or Scala
Implement and support Data Lake / Lakehouse architectures
Integrate data from legacy and modern platforms. Preferred postgres DB migration to Azure Cloud.
Expertise in handling Data migration using API's.
Apply data quality checks and support data governance basics (metadata, lineage, access controls)
Optimize Spark jobs for performance, reliability, and cost
Collaborate with cross-functional teams, Product owners, Admin teams, BSAs, Scrum & Project managers


Required Skills & Experience:


8+ years of experience in data engineering in azure using Azure Databricks.
Strong experience with Azure Data Lake, Databricks, ADF, Postgres.
Strong Data Engineering experience in Big Data (Cloudera)
Hands-on expertise in SQL, Python and PySpark or Scala
Proven experience with batch & real-time ingestion
Experience with Kafka and big data processing
Added good to have experience with:
Informatica PowerCenter
Informatica IDMC or equivalent cloud data integration tools
SQL Server, Oracle, Azure SQL
Cosmos DB
Unix/Linux
Maestro (or similar enterprise schedulers)
Data tools such as Precisely or equivalent


Added Advantage


Insurance domain knowledge (Policy, Claims, Billing, Underwriting, Actuarial, Regulatory data)
Experience with Azure DevOps (AzDO) and GitHub-based CI/CD
Practical usage of Copilot for data engineering and development productivity
Exposure to Generative AI / GenAI use cases in data platforms and analytics






Role Descriptions: Microsoft Azure (ADB/ADF)
Essential Skills: Microsoft Azure (ADB/ADF)

Job Tags

Local area

Similar Jobs

Braze

People Data Analyst Job at Braze

 ...Braze in making data driven people decisions. The People Data Analyst will work collaboratively to help teams garner insights on programs...  ...on reporting and analytics needs.Maintain current Workday and Looker dashboards, proactively building improvements that drive... 

American Logistics Authority

Entry-Level Freight Dispatcher (Remote) - Earn $2,200 to $3,000 Weekly ($114,400-$156,000 Annually) Job at American Logistics Authority

Entry-Level Freight Dispatcher (Remote) Earn $2,200 to $3,000 Weekly ($114,400$156,000 Annually) We are hiring motivated individuals to join our logistics team as Freight Dispatchers. This is a remote position with strong income potential. Candidates must already ...

BIGGBY COFFEE

Barista Job at BIGGBY COFFEE

 ...Cloud Park and Biggby Meijer Springboro Pike, as well as on in Monroe, OH near the outlets. This franchise is owned and operated by the Meyer Family, a 4th generation based Dayton family, with long held ties to the community that opened there first store, Biggby Dayton, on... 

Jackson Therapy Partners

LCSW - Full Time Position Job at Jackson Therapy Partners

 ...Job Details: Growing veterans clinic! Easy-going team that works collaboratively Hybrid work model with on-call weekends...  ...holidays per year. Minimum Requirements Masters degree in social work (MSW) from an accredited educational program Two years of... 

micro1

M&A Attorney Job at micro1

 ...About this role Role Title: M&A Attorney Role Type: Contractor Location: Remote We are seeking seasoned M&A attorneys for...  ...field. In this role, you will review, assess, and contribute to contract redlining workflows used to train and evaluate state-of-the-art...