Apache spark Sr developer

Position: Apache spark Sr developer


Location: Client Locations only: NYC/NJ, Boston, Pittsburgh only. (1 DAY
office and 4 day Remote). No 100% remote.



Job Summary :

- We are looking for a Spark developer who knows how to fully exploit
the potential of our Spark cluster.
- You will clean, transform, and analyze vast amounts of raw data from
various systems using Spark to provide ready-to-use data to our feature
developers and business analysts.
- This involves both ad-hoc requests as well as data pipelines that are
embedded in our production environment.
- Experience : 8 to 10yrs.
- Required Skills Technical Skills : Apache Spark ,Spark ,Scala
- Domain Skills : Nice to have skills
- Technical Skills : Junit
- Domain Skills : Asset Management Operations ,Fund Operations
- Technology : Data Management



Roles & Responsibilities :

- We are looking for a Spark developer who knows how to fully exploit
the potential of our Spark cluster.
- You will clean, transform, and analyze vast amounts of raw data from
various systems using Spark to provide ready-to-use data to our feature
developers and business analysts.
- This involves both ad-hoc requests as well as data pipelines that are
embedded in our production environment.
- Create Scala Spark jobs for data transformation and aggregation
- Produce unit tests for Spark transformations and helper methods
- Write Scala doc-style documentation with all code Design data
processing pipelines