OpenAIRE is hiring a Data Engineer!
We are looking for a developer with the profile of a Data Engineer to join our team. You will use various methods to transform raw data into useful data systems. You should have strong analytical skills and the ability to combine data from different sources. If you are detail-oriented, with excellent organizational skills and experience in this field, we’d like to hear from you.
About the job
OpenAIRE AMKE is a non-profit company operating as a service provider for the European research setting for at least 10 years. OpenAIRE is seeking a Java Developer with experience in building high-performing, scalable, enterprise-grade applications.
The OpenAIRE service infrastructure sits on a big data cluster where more than 500Mi records and close to 15Mi full-texts are collected, processed, and made accessible relying on cutting-edge data wrangling technologies and methods to build a knowledge graph known as the OpenAIRE Research Graph. Data engineering challenges are to be tackled at all levels of the Graph data flow:
● Collecting, validating, and storing metadata records and related files of different formats from thousands data sources;
● Deduplicating and interlinking metadata records;
● Enabling platforms to support full-text mining and deep learning techniques;
● Building enabling technologies to support the analysis of big data graphs;
● Scalable subscription and notification platforms to distribute Graph content to interested sources;
● Supporting efficient and user targeted search and discovery technologies.
Selected candidates will join forces with the OpenAIRE team located in Pisa - Italy, at “Consiglio Nazionale delle Ricerche”, but can carry out their activities mainly from remote locations. Candidates should however be available for traveling to meet with the technical team in Europe, up to 6 times per year, on request of the technical coordination.
The contract will last at least 2 years, with possible renewal if both parties are willing to continue, possibly starting from October 2021. Depending on the entry level skills, salaries may range from 42.000 EUR to 72.000 EUR per year (see minimum and favoured requirements below). The salary is intended to be gross, i.e. taxes, insurance, etc. are included, as well as expenses to travel to Pisa if working remotely. We do not issue or fund a working VISA for Europe.
Responsibilities will include
● Contribute to all phases of the development lifecycle
● Write well designed, testable, efficient code
● Ensure designs are in compliance with specifications
● Prepare and produce releases of software components
● Support continuous improvement by investigating alternatives and technologies and presenting these for architectural review
● Fluent written and spoken English language
● Team-work skills
● Proven hands-on Software Development experience
● Proven working experience in Java development
● Hands-on experience in designing and developing applications using Java EE platforms
● Object-Oriented Analysis and design using common design patterns.
● Good understanding of Relational Databases, SQL and ORM technologies (JPA2, Hibernate)
● Experience with test-driven development
● Proven experience with management of software projects using Maven and Version Control System (e.g., Git)
● Good knowledge of the architecture/commands of Unix-like systems
● Good knowledge of TCP/IP network (TCP, UDP, IPv4, IPv6, DNS)
● Foundation elements of distributed and parallel programming
● Knowledge of XML / XPATH / XSLT / XQUERY
● Experience in the Spring Framework
● Modeling and use of databases on NoSQL systems (e.g. Mongodb), and on systems for information retrieval (Solr, ElasticSearch)
● Big data processing solutions (Hadoop HDFS, Hive, GraphX, etc.)
- You will become a member of the OpenAIRE technical team and will lead/implement activities which will
- Analyze and organize raw data
- Build data systems and pipelines
- Evaluate business needs and objectives
- Prepare data for prescriptive and predictive modeling
- Combine raw information from different sources
- Explore ways to enhance data quality and reliability
- Identify opportunities for data acquisition
- Develop analytical tools and programs
- Work closely with the CTO and other team members on several projects
- Potentially represent OpenAIRE in European project technical meetings.
Qualifications, SKILLS and experience
- Experience in the implementation of algorithms on top of the Apache Spark framework - required
- Experience in modeling and use of databases on NoSQL systems (e.g. Mongodb, HBase), and on systems for information retrieval (Solr, ElasticSearch) - required
- Experience in using and applying
- Spring & SpringBoot Framework
- IoC Paradigm (e.g. Spring)
- MVC paradigm and framework to support web programming (e.g. Angular.js, Bootstrap, or similar)
- XML database (e.g. existDB)
- Previous experience and consolidated skills in
- Java and at least one other language (e.g. C / C ++ / C #, Perl, Python, Go)
- Web services (SOAP, REST), web applications, and application containers (e.g. Tomcat, Jetty, Docker)
- The architecture/commands of Unix-like systems
- SQL language to create and query databases
- Management of software projects with Maven and Version Control System (e.g., Git)
- Fluent written and spoken English language
- Excellent team-work skills
Terms of employment
- The position is offered for a period of one-year as an associate (in consultancy terms). It is renewable upon satisfactory performance.
- This may be a full-time or part-time position.
- Even though OpenAIRE offices are located in Athens, selected candidates will join forces with the OpenAIRE team located in Pisa, at Consiglio Nazionale delle Ricerche, but can carry out their activities mainly from remote.
- Candidates should be available for travelling to meet with the technical team in Europe, up to 6 times per year, on request of the technical coordination (paid travel).
- Depending on experience and country of residence, gross salary for a full-time position will be in the range of € 4,000 to € 6,500 per month.
An equal opportunity employer
Here at OpenAIRE we believe in equality & diversity. OpenAIRE ensures equal opportunities, treatment and access to all candidates regardless of their sex, race, colour, ethnic or social origin, genetic features, language, religion or belief, political or any other opinion, membership of a national minority, property, birth, disability, age or sexual orientation.
How to apply
Dates and deadlines
- Closing date for applications: Open till we find you!
- Online interviews: asap
- Expected starting date: asap