Renewable Energy Sources such as wind and solar do not emit CO2 but their production vary considerably depending on time and weather. Thus, it is important to use the flexibility in device loads to shift energy consumption to follow the production. For example, an Electrical Vehicle (EV) can be charged very flexibly between arriving home at 5PM and leaving again at 7AM. Utilizing all available energy flexibility requires applying machine learning and AI on massive amounts of Big Data from many different actors and devices, ranging from private consumers, over companies, to energy network operators, and using this to create digital solutions to enable and exploit flexibility.

The project Flexible Energy Denmark (FED) has built the foundation for this for the entire Danish society. Specifically, data is collected from a number of utilities (electricity, heat, water) throughout Denmark. The data is stored in the Danish National Energy Data Lake, hosted at Center Denmark, to enable efficient and advanced analysis. The Data Lake is built using only open source tools which can run both on-premise and in cloud settings.

In this talk, we describe the requirements for the data lake based on a representative case study, present its technical architecture, and provide a comparison of relevant tools along with the arguments for which ones we selected. Finally, recent developments related to Data Spaces are covered.