Warning: count(): Parameter must be an array or an object that implements Countable in /home/anton702/public_html/wp-includes/post-template.php on line 317
Data Engineer - IoT BigData Jobs

Data Engineer


National Grid is seeking a data engineer with specialization in web crawler skills to join their Advanced Data and Analytics team. This role will bring start-up challenges and opportunities inside a big and stable company.

The data engineer will develop web crawler applications to grow our data inventory for analytics and modeling. You will design and implement deep web crawling mechanisms to extract publicly available data to further develop the knowledge assets of National Grid. You embrace the challenge of unraveling raw data and providing innovative solutions to our data consumers. Incumbents should be prepared to work in a highly multi-tasked environment with rapidly changing business priorities. Abilities to work cross functionally and in an Agile Team Setting are a must.


  • Design and implement systems to either manually or automatically download data from websites and parse, clean and organize the data
  • Learn new data sources and determine how best to structure the data for use in advanced analyses
  • Research opportunities for data acquisition
  • Assess and resolve data quality issues and correct at source.
  • Design, construct, install, test and maintain highly scalable data management systems
  • Ensure all data solutions meet business requirements and industry practices
  • Integrate new data management technologies and software engineering tools into existing structures.
  • Have extensive experience in employing a variety of languages and tools to marry disparate data sources
  • Have knowledge of different database solutions (NoSQL or RDBMS)
  • Have knowledge of NoSQL solutions such as MongoDB, Cassandra, etc.
  • Build scalable data pipeline solutions in a cloud environment
  • Work effectively both in a local server environment and in a cloud-based environment
  • Collaborate with Data Architects and IT team members on project goals
  • Collaborate with Data Scientists and Quantitative Analysts
  • Communicate effectively and translate business requirements into data solutions


  • Master’s degree in a data intensive discipline (Computer Science, Applied mathematics or equivalent) is strongly preferred, with a background in “big data” computer programming and/or a minimum of 3-5 years experience in “big data” processing. Additional preference would be given to a candidate with a PhD degree in a data intensive discipline. Exceptional candidates considered with Bachelor’s degree or Master’s degree in progress.
  • Strong programming experience with: Python, Java, SQL, Ruby
  • Proven experience with web communication protocol and web crawling tools.
  • Proven experience with building and deploying ETL pipeline
  • Proven experience with emerging big data technologies
  • Proven experience with AWS
  • Experienced with querying NoSQL databases
  • Experienced with relational databases and SQL
  • Plus – having geospatial and GIS skills
  • Plus- experience with one or more specialize areas: image and remote sensing data, natural language data

Job : Cust & Market Analytics
Primary Location : NY-Brooklyn

Organisation : Cust & Market Analytics
Schedule : Full-time

Job Posting : Oct 3, 2016, 3:10:05 PM Unposting Date : Ongoing

To apply for this job please visit tinyurl.com.