Job Details
Job Code
PG Job Requisition - 511
Primary Skills
Python,SQL,scala,ETL,Databricks
Tax Terms
C2C,W-2,1099
Posted Date
02/21/2024
PG Job Requisition - 511 - DataBricks Developer // Remote
Remote Job - Yes United States | Posted - 02/21/2024
Company Overview
Follow us on Linkedin: https://www.linkedin.com/company/pulivarthigroup/
Pulivarthi Group LLC is a Global Staffing & IT Technology Solutions company, with our prime focus of providing world class solutions to our customers with the right talent. We combine the expertise of our team and the culture of your company to help you with the solution that is affordable and innovative using high quality standards and technologies.
We’ve served some of the largest healthcare, financial services, and government entities in the U.S.
Job Description
DataBricks Developer // Remote
Job Description:
Proficiency in working with the Databricks Unified Analytics Platform, including notebooks, clusters, jobs, and libraries.
Strong programming skills in languages commonly used with Databricks, such as Python, Scala, and SQL.
Experience in designing and implementing ETL processes within Databricks using Spark SQL, DataFrame API, and structured streaming.
Understanding and mastery of Spark DataFrames and Datasets for efficient and structured data manipulation.
Experience with structured streaming in Databricks for real-time data processing and analytics.
Proficiency in working with the Databricks Unified Analytics Platform, including notebooks, clusters, jobs, and libraries.
Strong programming skills in languages commonly used with Databricks, such as Python, Scala, and SQL.
Experience in designing and implementing ETL processes within Databricks using Spark SQL, DataFrame API, and structured streaming.
Understanding and mastery of Spark DataFrames and Datasets for efficient and structured data manipulation.
Experience with structured streaming in Databricks for real-time data processing and analytics.
Roles & Responsibilities:
Should be able to use version control systems, such as Git, to manage codebase changes and collaborate with a development team.
Should have Collaboration skills using Databricks notebooks, including sharing, versioning, and commenting on code.
monitor Databricks workloads, interpret logs, and optimize performance using built-in and external monitoring tools.
Create Scripts to automate tasks within Databricks, leveraging APIs and Databricks CLI.
Implement of cluster autoscaling in Databricks to optimize resource utilization.
Should be able to use version control systems, such as Git, to manage codebase changes and collaborate with a development team.
Should have Collaboration skills using Databricks notebooks, including sharing, versioning, and commenting on code.
monitor Databricks workloads, interpret logs, and optimize performance using built-in and external monitoring tools.
Create Scripts to automate tasks within Databricks, leveraging APIs and Databricks CLI.
Implement of cluster autoscaling in Databricks to optimize resource utilization.