10Pearls is a growing, energetic, and highly-reputed product development company that specializes in mobile apps, enterprise software, gamification and great user experiences.
Led by an experienced management team, and serving impressive clients, 10Pearls is seeking professionals with entrepreneurial spirits.
We seek professionals who thrive on new challenges. Our employees have the unique opportunity of not only helping solve challenges for our clients, but also to help define 10Pearls’ growth and direction.
Our unique business practices, culture and immense opportunity for growth help us attract professionals that have an entrepreneurial spirit.
We are an equal opportunity employer and are committed to maintaining a diverse workplace.
We are looking for a Data Engineer . Ideal candidate should have a Bachelor’s degree in Computer Science with 2 3 years of programming experience on Python, C / C++, Java, Perl, Golang, or other such languages.
Strong knowledge of database solutions is must.
Develop, construct, test and maintain optimal data pipeline architecture
Assemble large, complex data sets that meet functional / non-functional business requirements
Identify ways to improve data reliability, efficiency and quality
Prepare data for predictive and prescriptive modeling
Use data to discover tasks that can be automated
Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS big data’ technologies
Identify, design, and implement internal process improvements : automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
Monitoring processes performance and advising any necessary changes
Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
Create data tools for analytics that assist in building and optimizing the product into an innovative industry leader
Work with data and analytics experts to strive for greater functionality in our data systems.
Demonstrate proficiency in data management and automation on Spark, Hadoop, and HDFS environments
Understanding database management, and as such, in-depth knowledge of SQL is required. Likewise, other database solutions, such as Mongo, Cassandra or Bigtable, are good to have.
The candidate must have,
Good communication skills
Experience with Apache Hadoop, Hive, Spark, Airflow, Sqoop, Snowflake, Apache Livy, Delta Lake
Experience with streaming technologies such as kafka
Experience of working with programming languages like Scala, Java, SQL, Python, R etc.
Experience with IAC and automation frameworks such as Terraform, Ansible, Jenkins and Packer
Ability to dig deeper into the issues of the production critical systems and provide permanent fixes for the system
Experience in managing data in relational databases and developing ETL pipelines
Exposure to enterprise level service such as Cloudera, Databricks, AWS, etc
Exposure to AWS data services and technologies such as EC2, EMR, Kinesis, Lambda, DynamoDB are nice to have