Bigdata - Principal Architect

Job Description

Responsibilities:You will own the Data Platform technical customer engagements including architectural design sessions, specific implementation projects and/or MVPs. The ideal candidate will have experience in customer-facing roles and success leading deep technical architecture discussions with senior customer executives, Enterprise Architects, IT Management and Developers to drive Data Platform solutions to production.Key responsibilities include:Understand customers overall data estate, IT and business priorities and success measures to design implementation architectures and solutions.Apply technical knowledge to architect and administer solutions that meet business and IT needsEnsure that solution exhibits high levels of performance, security, scalability, maintainability, appropriate reusability and reliability upon deploymentCollaborate with other Cloud Solution Architects and Administrators in developing complex end-to-end Enterprise solutions on Microsoft Azure platform and Cloudera,Maintain technical skills and knowledge, keeping up to date with market trends and competitive insights; collaborate and share with the technical community while educate customers on Azure platform and ClouderaBe an Azure and Cloudera Data Platform evangelist with customers, partners and external communitiesTechnical experience:- Platform administration like Hadoop/Elastic Cluster, Cloudera Manager, Cloudera Director in Microsoft Azure cloud and on premise- Knowledge on Kerberization & Sentry Implementation- Understand the source systems data, gather & able to transfer the data into HDFS as RAW and Decomposed layer (including writing Oozie workflow, coordinators, Sqoop and Flume)- Exposure in handling huge quantities of data by taking advantage of both batch and speed methods- Strong knowledge of and experience with data wrangling; potentially other advanced math as well and good working knowledge of SQL query- Experience in processing large amounts of structured / unstructured data- Capability to design and document solutions independently, do code reviews, Ensure standard Quality Assurance activities are completed on time- Hands on with spark and kafka with Java/Scala/PythonMandatory Skills- Minimum 7 years of working experience in the Azure, Cloudera, Apache Hadoop framework, HDFS, Map Reduce, Hive, Flume, Sqoop, Oozie, Spark, Scala, Spark Streaming, Kafka and HBase- Minimum of 5 years of experience in Big Data and batch/real-time ingestion and analytical solutions leveraging transformational technologies- Good understanding about Lambda architecture- Implements security and recovery tools and techniques as required- Ensures all automated processes preserve data by managing the alignment of data availability and integration processes- Having exposure on Microsoft Azure Administration with Cloudera, Cloudera Director, Kerberization & Sentry- Having exposure on the Cloudera Manager, Cloudera Navigator, Hue & Shell Scripting- Having exposure on Microsoft Azure Cloud Migration from on premise to cloud- Kafka & Flume - Administration- Having exposure in automated deployment, monitoring and DevOps adoption- Having exposure on ADLS, Network Segmentation, Azure & on-premise network components, Azure Resources/cost optimization & Performance Optimization- Elastic Search & Elastic Search Cloud experience would be added advantage- Automated workflow migration would be added advantage- Integrating Hadoop with Oracle and NoSQL databasesCRITICAL COMPETENCIES- Technology Doers / Full stack- Drive Efficiency and leaders in cutting edge technologies- Technical Capability development- Transparency and Collaboration- Self-driven and Self-learning- Problem solving and decision making- End-End solution- Appetite to develop in multiple platformsPreferred qualificationsCloud Certifications in one or more of the following: AWS, Azure, GCPMiddleware Certifications: IBM WebSphere, IBM MQ, Tomcat, Apache web server.DevOps Tools Certifications: Jenkins, Chef Certification, Puppet.Five or more years experience in WebSphere, Resin, or another Java application server platform(s), as well as MQ, DataPower (or other XML gateway)Experience using Chef to develop, code, test, and debug new Chef cookbooks and recipes or enhancements to existing software.Certified Scrum Developer (CSD)Hands-on experience with Big Data Applications and have working experience with 5 or more: Hadoop, MapReduce, HDFS, YARN, Cloudera, Ambari, HortonWorks, Hadoop HDFS, Accumulo, Spark, Streaming, Kafka, Cassandra, elastic Search.Experience with Hadoop architecture, administration and database managementExperience maintaining Hadoop clusters.Experience sizing and scaling clusters, adding and removing nodes, provisioning resources for jobs, job maintenance and schedulingStrong understanding of JVM debugging, management and tuningOperational experience with Hadoop technologies include Ambari, Ranger, Atlas, Knox, NiFi, Kafka, Storm, Hive, Pig, MapReduce, HDFS, HBase, Accumulo, SparkExperience with relational databases (preferably Microsft SQL Server) and data warehouse conceptsExperience with MDM Infosphere/Datastage/Informatica in a Hadoop environment.Familiarity with Java/scala programmingSoft SkillsRelationship Building. Proven track record of building deep technical relationships with senior IT executives in large or highly strategic accounts. Experience in managing various stakeholder relationships to get consensus on solution/projects. requiredProblem Solving. Ability to solve customer problems through cloud technologies requiredCollaboration and Communication. Acknowledged for driving decisions collaboratively, resolving conflicts and ensuring follow through with exceptional verbal and written communication skills. Ability to orchestrate, lead, and influence virtual teams, ensuring successful implementation of customer projects.Presentation skills with a high degree of comfort with both large and small audiences (Senior Executives, IT management, Database administrators and Data Scientist) requiredEnterprise-scale technical experience with cloud and hybrid infrastructures, architecture designs, database migrations, and technology management. requiredExpertise in data estate workloads like HDInsight, Hadoop, Cloudera, Spark, Python, Kafka, Java requiredTechnical Aptitude and experience to learn new technologies and understand relevant cloud trend requiredCompetitive Landscape: Knowledge of cloud development platforms preferredPartners: Understanding of partner ecosystems and the ability to leverage partner solutions to solve customer needs preferred

Keyskills :
Bigdata - Principal Architect