Must have skills:
Mandatory and candidate need to be strong with Spark, Scala, AWS, AWS (EMR, S3).

1. Handson experience of Spark, Scala and conversant with SQL
2. Good knowledge on Hadoop (Oozie)
3. Reverse engineer the SQL queries, Scala code to understand functionality
4. Capable of identifying, analysing and interpret patterns and trends in complex data sets
5. Should have strong experience on AWS (EMR, S3)
6. Has worked on creating database design, data models & techniques for data mining.

Good to have skills:
1. Adept at using data processing platforms like Hadoop and CouchDB
2. Performing analysis to assess quality and meaning of data
3. Preparing reports for the management stating trends, patterns, and predictions using relevant data

Detailed Requirements:
1. Using automated tools to extract data from primary and secondary sources
2. Removing corrupted data and fixing coding errors and related problems
3. Performing analysis to assess quality and meaning of data
4. Filter Data by reviewing reports and performance indicators to identify and correct code problems
5. Using statistical tools to identify, analyze, and interpret patterns and trends in complex data sets that could be helpful for the diagnosis and 6. prediction
7. Assigning numerical value to essential business functions so that business performance can be assessed and compared over periods of time.
8. Analyzing local, national, and global trends that impact both the organization and the industry
9. Preparing reports for the management stating trends, patterns, and predictions using relevant data
10. Working with programmers, engineers, and management heads to identify process improvement opportunities, propose system modifications, and devise data governance strategies.
11. Preparing final analysis reports for the stakeholders to understand the data-analysis steps, enabling them to take important decisions based on various facts and trends.
12. Knowledge of programming languages like SQL, Oracle, R and Python
13. Technical proficiency regarding database design development, data models, techniques for data mining, and segmentation.
14. Experience in handling reporting packages like Business Objects, programming (JavaScript, XML, or ETL frameworks), databases
15. Proficiency in statistics and statistical packages like Excel, SPSS, SAS to be used for data set analyzing
16. Adept at using data processing platforms like Hadoop and Apache Spark
17. Knowledge of data visualization software like Tableau, Qlik
18. Knowledge of how to create and apply the most accurate algorithms to datasets in order to find solutions
19. Data mapping (ACRS, MARSHA) creation & maintenance, track open questions, issues, and support on all ongoing epics with data related queries