Node.Digital

Software Engineer, Databricks

Node.Digital

full-time

Posted on:

Origin:  • 🇺🇸 United States • Virginia

Visit company website
✨ AI Apply
Apply

Job Level

SeniorLead

Tech Stack

Amazon RedshiftAWSCloudElasticSearchNoSQLPostgresTypeScript

About the role

  • Supporting teams to migrate services, applications and platforms from legacy back-end systems to Databricks.
  • Identifying the optimal path for migration, building the plan for migration and finally, execution of the plan.
  • Data Pipeline migration of legacy pipelines from NiFi to Databricks, complete with validation.
  • Implementing the medallion model for each of the data assets being migrated to Databricks.
  • Develop an SOP for integration of data assets into the Databricks platform focused on efficiency, instrumentation and performance.
  • Optimize development, testing, monitoring and security for data assets being added to the Databricks platform.
  • Develop and implement a strategy for optimizing migration and integration of data assets to the Databricks platform.
  • Develop code using various programming and scripting languages to automate/optimize data ingestion, pipeline orchestration and improve data management processes.
  • Ingest transparency, leveraging technologies such as AWS CloudWatch to identify places for measuring and gathering performance information on automated data pipelines
  • Ensure Data Engineering Team Standard Operating Procedures are appropriately captured and communicated across the team.
  • Ensure technical correctness, timeliness and quality of delivery for the team.
  • Demonstrate excellent oral and written communication skills to all levels of management and the customer.

Requirements

  • Must be a US Citizen
  • Active Secret (S) clearance. Must be able to obtain a TS/SCI clearance
  • Must be able to obtain DHS Suitability
  • 8 + years of directly relevant software development experience required.
  • Minimum of 5 years of experience performing data engineering work in a cloud environment.
  • Experience with relational, noSQL and/or file-based storage (e.g. Databricks, Elastic, Postgres, S3, Athena, etc.)
  • Experience working in a CICD Pipeline factory environment
  • Working knowledge of Databricks, Cloud Relational Database Services, NiFi, AWS Redshift and Elasticsearch
  • Bachelor's degree in Software Engineering, Computer Science or a related discipline is required. [Ten (10) years of experience (for a total of six (18) or more years) may be substituted for a degree.]
Analytica

Product Owner

Analytica
Mid · Seniorfull-timeDistrict of Columbia · 🇺🇸 United States
Posted: 25 days agoSource: analyticallc.applytojob.com
Amazon RedshiftAWSAzureBigQueryCloudCyber SecurityOracleSQL
eSimplicity

Senior DevOps Engineer

eSimplicity
Seniorfull-time$106k–$137k / year🇺🇸 United States
Posted: 15 days agoSource: recruiting.paylocity.com
Amazon RedshiftAWSCloudDjangoDockerHadoopJavaJavaScriptMavenPythonTerraform
Common Securitization Solutions

Senior Data/AI Engineer

Common Securitization Solutions
Seniorfull-time$142k–$165k / year🇺🇸 United States
Posted: 30 days agoSource: careers-usfintech.icims.com
Amazon RedshiftAWSCloudDistributed SystemsETLKafkaPythonPyTorchScikit-LearnSQLTensorflow
Tiger Analytics

Cloud Data Architect

Tiger Analytics
Senior · Leadfull-time🇺🇸 United States
Posted: 18 hours agoSource: apply.workable.com
Amazon RedshiftAWSAzureBigQueryCloudFirewallsGoogle Cloud PlatformJavaKubernetesNoSQLPythonScala+2 more
Secure it Securities Corp

Sales Engineer

Secure it Securities Corp
Mid · Seniorfull-time$225k–$300k / yearFlorida, North Carolina, Tennessee · 🇺🇸 United States
Posted: 27 days agoSource: securiti.freshteam.com
Amazon RedshiftAWSAzureCassandraCloudDynamoDBGoogle Cloud PlatformHBaseHDFSKubernetesLinuxMongoDB+1 more