Principal Data Engineer

Principal Data Engineer

Redwood City, CA 94063 2016-10-27 - –

nominum Nominum

SUMMARY:

Reporting to the Sr Director of Engineering, the Principal Data Engineer is responsible for architecture and development of our data infrastructure systems. Nominum processes over 100 billion pieces of data per day. Nominum uses this data to identify and mitigate security threats, stop distributed denial of service attacks, and generate actionable insights at a scale beyond what most companies are facing today. The Principal Data Engineer will be responsible for building infrastructure and tools that analyze a variety of data sets, process them to identify security threats and other data-driven classifications, and publish extracted results in to ISP networks around the world.

RESPONSIBILITIES:

? Become an expert in our data sets, both raw streaming DNS data and our 3rd party list data

? Develop and implement the infrastructure to move large amounts of data in and out or our Hadoop cluster, with the use of Kafka

? Develop the tools to aid in the extraction of meaningful insights from the large amount data we work with

? Work with our third party list providers to clean and improve the accuracy of the data

? Design and implementation of tools used in the analysis of data and automate the process of identifying threat vectors in real-time, streaming data

? Interact with the other development teams to determine the data required for our analytics applications and reporting

QUALIFICATIONS:

? Expertise with Java, C++ or other OO language, with knowledge of Python

? Solid understanding of data streaming and warehousing systems, such as Hadoop, HDFS, Kafka

? Expert knowledge of database systems (MySQL, NoSQL, etc.)

? A solid understanding of JSON, Avro and REST APIs

? Experience with any one of the following a plus (text mining, natural language processing, information retrieval, etc.)

? Experience with UNIX/Linux systems desirable

? Experience with network security and identifying network vulnerabilities a plus

? Experience with web crawling/scraping technologies desirable

? Outstanding organizational, analytical and problem solving skills

? Clear written and oral communication skills

Education and/or Experience:

? BA/BS/MS or equivalent experience in Computer Science, Engineering or related technical field

? 8+ years of experience in software design and development

? Graduate level data analytics/research desirable

Supervisor Responsibilities

N/A

To apply for this job please visit tinyurl.com.