Architecture
Overview
Logprep is designed to receive, process and forward log messages. It consists of several interconnected components that work together to make this possible.
Starting Logprep
The following diagramm shows the starting behaviour of Logprep.
Pipeline Manager
This diagram shows the creation of Multiprocessing Pipelines and the shutdown of failed Pipelines.
Pipeline
This diagram shows the flow of the Pipeline. The starting-point is the creating of the PipelineManager and therefore the start of the MultiprocessingPipeline.
Input
In this diagram, some parts are specific for the ConfluentKafkaInput Connector. These was deemed to be important enough to be part of the diagram.
Processor
Below is a visualization of all available processors of Logprep. These diagrams also show which processors inherit from what. The first of these diagrams describes the process up to the actual application of the rule that is implemented in the respective processors.
Ruletree
The Ruletree diagramm shows how the matching rules for a given event are searched for and found.
Output
In this diagram, the last part about the backlog is specific for the Elasticsearch/ Opensearch Output. This was deemed to be important enough to be part of the diagram.
Event flow
The following diagrams illustrate the flow of a single event to make it more comprehensible.
Multiprocessing
This diagram shows what ressources are shared within the multiprocessing processes and how the processes are started and stopped.