Task 8: Deploying DER Applications
Overview
This document outlines the deployment and functionality of the five core applications required for the final task. The system is designed to simulate a real-world solar power environment by generating mock inverter data, streaming it through a robust data pipeline, and making it available via the SunSpec Modbus protocol.
The primary data flow is as follows: The Streamer application generates time-series data and sends it to a Kafka topic. This data is then consumed and used to update the values of a simulated Inverter. Finally, a SunSpec Client can connect to the inverter to read the real-time values, confirming the end-to-end data flow.
Applications to deploy
Zookeeper
Purpose: Acts as a coordination service for the Kafka cluster. It is a mandatory prerequisite for running Kafka.
Role: Zookeeper manages the brokers in the Kafka cluster, keeps track of the status of Kafka nodes, and maintains topic configuration information. It ensures the reliability and stability of the Kafka service.
Kafka
Purpose: A high-throughput, distributed messaging system. It serves as the central data bus for our simulation.
Role: Kafka receives data messages from the Streamer application (the producer) and stores them durably in a topic. The service responsible for updating the inverter will then consume these messages from the topic.
Streamer
Purpose: A data producer application that simulates the output of a real solar inverter.
Role: The Streamer generates realistic, time-series data points (e.g., AC Power, DC Voltage, Current) and publishes them as messages to a specific topic in the Kafka cluster. This application is the starting point of our data pipeline.
Inverter
Purpose: A simulator that emulates a real-world, SunSpec-compliant solar inverter.
Role: This application listens for new data from the Kafka topic. Upon receiving a new message, it updates its internal Modbus register values. This makes the inverter appear as if it is a live, functioning device generating data in real
time. It effectively acts as the primary data consumer in the pipeline.
SunSpec Client
Purpose: A utility to connect to and read data from a SunSpec-compliant device.
Role: The client is used for verification and monitoring. It connects to the Inverter simulation using the Modbus protocol to read its register values. This allows us to confirm that the data is flowing correctly from the Streamer, through Kafka, and is being accurately reflected in the inverter's state.
Deployment Order
The applications must be deployed in the following sequence to ensure proper initialization and communication:
Zookeeper: Deploy first. It is essential to check its logs to confirm it has started successfully before proceeding.
Kafka: Deploy after Zookeeper is running.
Streamer: Deploy after Kafka is running.
Inverter: Deploy after the Streamer.
SunSpec Client: Deploy last to verify the end-to-end data flow.
Deployment Targets
k3smain: zookeeper, kafka. streamer Nano: inverter AGX: Sunspec Client
Group members are free to pick whichever application they would like to deploy, but the overall order of deployment must be maintained. The tasks must be performed from the control plane. If the group members decide to do this as a group task, you may enter a shared terminal (screen -x group_session). If not, they can be performed individually. Each application must be deployed only once.