Data Nugget May 2024

Mark your calendars as DAMA Norway completes its five years of excellence and commitment to providing valuable insights in the data management domain.

Welcome to the May edition of the data nugget. Our coverage of stories this month offers a comprehensive perspective on how technological developments are revolutionizing various industries, and transforming the way organizations manage, protect, and leverage their data. Join us as we explore the latest trends and insights that are driving the future of data management.

First, we invite you humbly to the DAMA Norway's fifth anniversary. Second, we have a review of the opportunities and risks associated with an AI-driven future. Third, we have an interesting read about the evolution of data engineering. And last but not least, we have the next podcast on the 'datafication' of public administration.

Happy reading!

Let's grow Data Nugget together. Forward it to a friend. They can sign up here to get a fresh version of Data Nugget on the last day of every month.

DAMA Norway celebrates 5 years of excellence

Nugget by Siri Granerød

In 2024, DAMA Norway celebrates five years as a chapter of DAMA. It has been a remarkable journey since our humble beginnings, and we look forward to reflecting on our accomplishments and exploring the road ahead for data management and data governance.

We invite our community to celebrate DAMA Norway’s anniversary with great joy and pride. in this celebration, we take a look back on the past years and also set a focus on the challenges: Why do many data management projects fail?

To celebrate, some of our greatest contributors like Ole Olesen-Bagneux and Winfried Adalbert Etzel will share their knowledge about the challenges and solutions to successfully make organizations data-driven. We have also reached out to another important organization, the Association of Change Management Professionals where ACMP Norway's President Agnes Beathe Steen Fosse will provide input on how change management may contribute to successful data management initiatives.

Hope to see as many as possible in our event: Endringsledelse: Den hemmelige ingrediensen til Data Governance

On site in Skøyen Atrium Oslo, June 7th at 13.00. Hope to see you there and celebrate with us!

Balancing AI Innovation and Sustainability

Nugget by Achillefs Tsitsonis

The pace of innovation in Artificial Intelligence and Machine Learning this last year has been an unprecedented one. Since the launch of ChatGPT from Open AI, the number of investments made in the space has increased exponentially. As the race towards Artificial General Intelligence and other AI applications intensifies, we notice that all such investments carry a significant burden, which is energy-related needs. Compute is the new currency, and both corporations as well as nations are going to compete to secure it.

Where does that leave us though in terms of our climate goals and sustainability principles and how can AI development balance both the pace of innovation and sustainable development?

There is no question that AI can contribute positively to fighting climate change. One of the main challenges in this field is overcoming the immense job of discovering new and viable models for which possible solutions can exist. Machine Learning models are, however, much better suited to analyze both high-volume and high-complexity datasets. As the energy requirements for AI development increase, predictive energy allocation models can help us use our energy resources more efficiently and provide us even with new ways of producing energy that we are not aware of yet.

Furthermore, one other major area where AI can contribute positively is climate adaptation and resilience. As climate change shows its effects, it becomes harder to predict how, when and where these effects manifest. From early warning systems based on predictive analytics to biodiversity monitoring, we must acknowledge the positive impact that AI applications have.

Despite all of the above, there are also significant risks associated with AI development as well. Electricity-producing capabilities will become one of the limiting factors for further development, which brings up questions of what needs to be sacrificed to satisfy these needs. Water is another critical element for powering but mostly cooling data centers for AI development. For example, according to research from Cornell University, training GPT-3 in Microsoft's state-of-the-art U.S. data centers can directly evaporate 700,000 litres of clean freshwater. These needs are only increasing as time goes by, so how do we expect to tackle them?

Some possible solutions could focus on:

• Establishing measurement standards for Machine Learning models.
• Expanding data collection in areas that can contribute to sustainability improvements.
• Identifying AI-specific impacts, both positive and negative, boosts proactive instead of reactive action.
• Looking beyond operational energy use and emissions and identifying other related climate impacts

We need to focus on achieving our AI goals as a society but while doing that, we have an opportunity and a moral obligation to contribute positively to ensuring a sustainable future for our kids and all those that will take over our planet from us.

The fanciest title of all – Data Engineer

Nugget by Gaurav Sood

Imagine that up until Facebook/X/Instagram was founded, the terms Data Engineer and Data Scientists were not heard off. All these companies that thrive on Data and make all their revenue from data are merely a decade old. And so is the relative use of Data Engineering and Data Science as buzzwords.

In the days before social media, people were still doing data mining, data manipulation and deriving meaningful insights from raw data. But instead of being called Data Engineers/Data Scientists, they were just statisticians, economists or computer geeks looking into the same part of the data science/engineering field as we know it today.

History of Data Engineering

Up until 2010, the data guys were usually divided into two categories: database developers (DBAs) and data warehouse developers (DWHs) based on the system they were working on. DBAs were mainly responsible for all the administration-level activities on the Database and server level, while the DWH developers were responsible for all the ETL and business logic implementation. Depending on the software, sometimes the DBA was a must because of the complexity involved in setting up the environment, while in other cases where it was relatively off the shelf, a DWH developer could handle most of the stuff.

Present and Future of Data Engineering

What does a data engineer do today? While the job is quite like the earlier days, a data engineer is a combination of both the DBA and the DWH developer. With the emergence of streaming data, web applications and cloud computing, a data engineer is expected to work in the backend database and networking level as well as understand the business logic and develop pipelines for ETL and reporting. More and more companies are moving towards Data Engineering roles and responsibilities and away from the traditional DBA-type roles. A big part of this transition is also because of the various cloud application models, IaaS, PaaS and SaaS, which give the companies an option to have everything managed from outside them and just focus on the business.

At the same time, companies are not investing time and money in building data warehouses, but they apply these data warehousing techniques in setting up the data pipelines, and, with the help of the data engineers, provide both the business intelligence and the science out of their data. These changes did not happen only due to cost-cutting or eagerness to have a faster result, but they are supported by the increase of open-source software and by the combination of traditional software engineering skills with data processing skills.

You can read more here. 

MetaDAMA 2#19: Datafication of Public Administration

Nugget by Winfried Adalbert Etzel

"It's about taking a step back to ask yourself: Should we even have a data-driven system for this?" (Det handler om å ta et steg tilbake for å spørre seg: Skal vi i det hele tatt ha et data-drevent system på dette her?)

We finish season 2 with a thought-provoking episode to maybe start some debate about data-driven public administration.

Lisa Reutter is a PostDoc at the University of Copenhagen (Københavns Universitet) connected to a project called Datafied Living. We talked about the importance of Social Science in Data and how data is intertwined with our lives. Lisa is researching in the field of critical data and algorithm studies at the interplay between tech, data and society.

Here are my key takeaways:

Data in Public Administration

• For a modern state to function properly and to ensure citizen rights, services, security etc. are provided, the state needs data.
• Data Management by the state for its citizens is not a new concept but has a long historical foundation.
• During the last years, we have used more, different and new data in administrative processes and enhanced technological development and a toolbox to derive value from data.
• Public administration has had a monopoly over the management and ownership of citizen data. But this has been challenged by private companies.
• Data-driven systems in the public sector are not there for profit, but to create value for society. Therefore, they need to be built to enhance our democratic values.

Registers

• Norway and other Scandinavian countries have established national registers to manage and administer society.
• There is a reason why registers are not unified in Norway, and this is to ensure a balance of powers.
• The opposite example of what can happen if a state collects information on its citizens without boundaries is to be found in the GDR (Eastern Germany)
• If all data of all aspects of your life are collected in one place, it is really easy to misuse these data.
• Through data, a state could see, predict, and control the behaviors of citizens. 

The public debate about data-driven

• Discussions can and should be about what data we are collecting, where we store data, what are we using data for, who could and should have access to that data, etc.
• Even with public debate about data use in public administration, limits and boundaries can never be defined clearly. Also, because this is individual and relative to context.
• Datafication is a political act. The citizens need to be involved in the process of technological advancement and intelligent use of data.
• The debate around data-driven public administration in Norway has not included the public actively.

Customer-centric vs. data-as-an-asset vs. democratizing data

• Is there a rhetorical ambiguity between being customer-centric and data-as-an-asset?
• Data democratization demands that citizens must use their time, resources and energy to ensure that public administration is working correctly.
• Is making data available leading to commercial parties capitalizing on that data and building solutions rather than creating transparency for citizens?
• The right education and skills are important, but they need to be available and attainable for all parts of society.
• Data Literacy is an own subject that is in dispute about what it should contain.

We need to understand, that this has implications on how we:

1. 1. Trust in the state
2. Trade - what do I give my data for? What do I get in return?
3. Build in accepted ways
4. Weight opportunities against risk
5. Ensure that the responsibility for understanding does not lie with the citizen alone
6. Gain knowledge and how everyone can get it.
7. Should invite for debate.

You can listen to the podcast  here  or on any of the common streaming services (Apple Podcast, Google Podcast, Spotify,  etc.) Note: The podcasts in our monthly newsletters are behind the actual airtime of the MetaDAMA podcast series.

Thank you for reading this edition of Data Nugget. We hope you liked it.

Data Nugget was delivered with a vision, zeal and courage from the editors and the collaborators.

You can visit our website here, or write us at dama@dnd.no.

I would love to hear your feedback and ideas.

Nazia Qureshi

Data Nugget Head Editor