Herman Jakobsen

Tech Lead @ Statnett · Senior Software Engineer @ Systek

Systek’s Herman Kolstad Jakobsen works as a consultant and has several years of experience as a Tech Lead at Statnett. At Statnett, Herman works with mission-critical real-time data, where technologies such as Apache Kafka and Databricks play a central role. Previously, Herman has given presentations at Statnett’s #techday conference, at «Den Norske Dataforening» and at JavaZone, where he shares his enthusiasm for DevOps and Apache Kafka. Herman has also contributed upstreams to the Apache Kafka Streams library.

Talk:
Controlling the Norwegian Power Grid in Real Time with Apache Kafka

In March 2025, the Norwegian Transmission System Operator, Statnett, implemented its biggest operational and digital change in 30 years! By shifting from manual, 60-minute balancing to automated activation with 15-minute intervals, the digital transformation ensures a more efficient, resilient, and stable electricity grid. And the backbone of it all: Apache Kafka!

We will take both an architectural and client-side dive into how Apache Kafka is used in the control loop of the Norwegian power grid, and explore how Kafka is used not only as an integration between systems, but also as a platform for streaming and aggregating data in real-time. All of this while still meeting strict requirements for robustness, observability, availability and fault-tolerance.

Key Takeaways:

  • Knowledge of how the power grid is controlled
  • How Kafka can be used to integrate systems, and stream and aggregate data in real-time
  • How to design robust value chains that meets consumers’ different requirements of real-time data vs historical correct data (i.e. handle delayed data, backfilling etc)
  • How to monitor and gain observability in a complex and long value chain
  • If time, how to design efficient, simple and robust Kafka Streams clients (frankly, I think this will be skipped).

Target Audience:

  • Everyone interested in how Apache Kafka is used in real high-stake use cases. Developers interested in how to design robust value chains and Kafka Streams clients.