Avangrid Senior Data Engineer in New Haven, Connecticut
Senior Data Engineer
Company: AVANGRID SERVICE COMPANY
New Haven, CT, US Rochester, NY, US Augusta, ME, US
Iberdrola is a global energy leader, the number one producer of wind power, and one of the world's biggest electricity utilities in terms of market capitalisation. The group supplies energy to almost 100 million people in dozens of countries including Spain, the United Kingdom (ScottishPower), the United States (AVANGRID), Brazil (Neoenergia), Mexico, Germany, Portugal, Italy and France, with a workforce of more than 35,000 people.
Iberdrola is leading the transition towards a sustainable energy model through its investments in renewable energy, smart grids, large-scale energy storage and digital transformation, to offer its customers the most advanced products and services. Thanks to its commitment to clean energy, Iberdrola is one of the companies with the lowest emissions and an international benchmark for its contribution to sustainability and the fight against climate change.
The purpose of this role is to work closely with the business and other IT team member to develop analytics solutions that help them understand, model and govern the company’s data and to ensure best practice is applied in the transit and transformation of data through its journey from creation to actionable information.
This role will also assist the business in the development and management of Enterprise Information Models and best practice Data Modeling and Data Architecture to support Business Intelligence and Data Mining initiatives across AVANGRID.
This role will be able to identify, manage and design requirements for data extractions, data transformations and data loading, to and from various technical platforms; including SAS, Oracle Exadata and Hadoop and other NoSQL ecosystems to ensure business value is delivered in the
most efficient way
•Analyze the existing data repositories and business processes to derive data profiles and identify data sources.
•Identify, design and maintain data models as required.
•Maintain production and development big data platforms along with the analytics applications.
•Assist with the definition of use cases and business use of Big Data Platforms such as Hadoop and other NoSQL platforms describing the advantages and disadvantages of options available
•Assist in the design of data acquisition, cleansing, and transformation into target data structures from potentially multiple sources such as SQL and non-SQL databases, SAP, structured and non-structured files, Hadoop.
•Design and develop programs that will run in Big Data infrastructures: On-premise cluster (Hortonworks), Cloud (MS Azure HDInsight):
•Automate the big data jobs that runs both on batch node and in real-time.
•Massive data processing (smart meter, transformer and other utility assets data)
•Operationalize analytic models to run efficiently (analytics on Hadoop)
•Data Provisioning (including ETL/ET)
•Present to business, non-technical audience being able to create, prototypes, and demonstrations to help management better understand the work.
•Help maintain documentation of data models and/or other reusable knowledge assets regarding data governance.
•Support in when and how to leverage new Big Data technologies such as, NoSQL, Hadoop and foster its usage within the AVANGRID organization with clear business outcomes.
Skills and Requirements
•Strong experience working with large data systems preferably Hadoop clusters or cloud based big data solutions (MS Azure HDInsight), developing and maintaining analytics applications
•Working knowledge of creating large data pipelines coming from different data source systems inside and outside of the organization.
•Write efficient code, queries and perform different testing mechanisms to identify the best code that can be run on clusters.
•Expert in choosing right technology that decrease costs and increase productivity.
•Experience in presenting solutions to larger audience including Managers, Directors, CIO etc.
•Strong familiarity with data management disciplines and tooling.
•Experience working in Big Data technologies such as Hadoop, Spark, Hive, Sqoop, Kafka, NiFi, HBase, CDP and other tools of big data ecosystem is desired.
• Expert in performing SQL jobs and familiarity of NoSQL databases (Document databases such as MongoDB, Column databases such as Cassandra) preferred.
•Knowledge in Zeppelin, Linux environment, etc. to quickly get the answer for business questions
•Working knowledge of BI tools (MicroStrategy, PowerBI)
•Should have experience in managing and maintaining the production applications and resolve the issues quickly.
•Experience in automating the big data jobs using tools such as Oozie.
•Should have experience in troubleshooting the application issues and fixing them as they come up.
•Expertise working with APIs.
•Knowledge of different data file formats like Parquet, ORC, avro, csv, etc
•Expertise in converting unstructured, semi-structured data into structured format which will be used to mine the information to solve business requirements
•Knowledge of working in environments that apply agile development best practices particulary SCRUM -Agile
•Strong commitment to establish Standards of Best Practices.
•Knowledge of SAP data structures (CRM and Utilities) would be an advantage.
•You will have the opportunity to work on a variety of digital projects and working closely with IT technical teams, Avangrid Architecture Team and Business teams.
•Work closely with AVANGRID Architecture team to evolve IT standard and architecture models as required.
•Engage Applications Managers as needed to gather requirement and manage expectations
•Establish and maintain relationships with 3rd party suppliers for researching activities.
Education & Experience Required:
BA/BS data analytics/computer science/information or similar degree is required. New grads and/or grads with 4 to 5 years of job related experience
•Good oral and written communication skills.
•Strong problem solving and analytical skills.
•Ability to multi task.
•Strong Interpersonal and Communication Skills.
•Willingness to travel.
•Must be a proven team player to work, promote and consolidate efficient team working relationships
Develop Self & Others
Empower to grow
Collaborate and Share
Be a role model
Focus to achieve results
Avangrid employees may be assigned a system emergency role and in the event of a system emergency, may be required to work outside of their regular schedule/job duties. This is applicable to employees that will work in Connecticut, Maine, Massachusetts, and New York within AVANGRD Network and Corporate functions. This does not include those that will work for Avangrid Renewables
AVANGRID’s employment practices and policies are geared to hiring a diverse workforce and sustaining an inclusive culture. At AVANGRID we provide fair and equal employment and advancement opportunities for all employees and candidates regardless of race, color, religion, national origin, gender, sexual orientation, age, marital status, disability, protected veteran status or any other status protected by federal, state, or local law. Learn more about equal employment by following this link (https://www.eeoc.gov/employers/upload/poster_screen_reader_optimized.pdf)
If you are an individual with a disability or a disabled veteran who is unable to use our online tool to search for or to apply for jobs, you may request a reasonable accommodation by contacting our Human Resources department at 203-499-2777 or email@example.com