Kubernetes Example Deployment

For this example, we need a working kubernetes cluster. Here we will use minikube, but every other kubernetes environment should do the job.

Setup Minikube

To install minikube, helm and kubectl follow the instructions below.

If you have docker already installed, you can install the needed components and start minikube with the following commands:

Install package prerequisites

sudo apt-get install -y \
    apt-transport-https \
    ca-certificates \
    curl \
    software-properties-common

Install minikube

sudo curl -Lo /usr/local/bin/minikube \
  https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64

sudo chmod +x /usr/local/bin/minikube

Install kubectl

sudo curl -Lo /usr/local/bin/kubectl \
  "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"

sudo chmod +x /usr/local/bin/kubectl

Install helm

wget https://get.helm.sh/helm-v3.15.1-linux-amd64.tar.gz
tar xzvf helm-v3.15.1-linux-amd64.tar.gz
sudo mv linux-amd64/helm /usr/local/bin/helm
sudo chmod +x /usr/local/bin/helm

add helm repositories

helm repo add bitnami https://charts.bitnami.com/bitnami

Configure and start minikube

minikube config set driver docker
minikube config set cpus 16
minikube config set memory 16GB
minikube addons enable ingress
minikube start

Deploy the example

The following steps install the actual opensiem example on the minikube cluster. It will install

At first you have to install the prometheus PodMonitor CRD:

Install the prometheus PodMonitor CRD

kubectl apply -f https://raw.githubusercontent.com/prometheus-community/helm-charts/main/charts/kube-prometheus-stack/charts/crds/crds/crd-podmonitors.yaml

Then you have to update and build the helm subcharts repository:

Add the bitnami helm repository

helm dependencies update ./examples/k8s
helm dependencies build ./examples/k8s

Next you are ready to install the opensiem example using:

Install opensiem

helm install opensiem examples/k8s

Make the cluster locally resolvable:

add hosts entry to resolve the cluster

echo "$( minikube ip ) connector.opensiem dashboards.opensiem grafana.opensiem" | sudo tee -a /etc/hosts

Test the defined ingresses:

Test the opensiem example ingress

curl -v http://connector.opensiem/health
curl -v http://dashboards.opensiem

Test the opensiem connector:

Test the opensiem example connector

❯ logprep generate http --input-dir ./examples/exampledata/input_logdata/ --target-url http://connector.opensiem --events 100 --batch-size 10

2024-07-17 11:15:35 301643 Generator  INFO    : Log level set to 'NOTSET'
2024-07-17 11:15:35 301643 Generator  INFO    : Started Data Processing
2024-07-17 11:15:35 301643 Input      INFO    : Reading input dataset and creating temporary event collections in: '/tmp/logprep_a51e1vh6'
2024-07-17 11:15:35 301643 Input      INFO    : Preparing data took: 0.0042 seconds
2024-07-17 11:15:35 301643 Input      INFO    : Cleaned up temp dir: '/tmp/logprep_a51e1vh6'
2024-07-17 11:15:35 301643 Generator  INFO    : Completed with following statistics: {
    "Number of failed events": 0,
    "Number of successfull events": 100,
    "Requests Connection Errors": 0,
    "Requests Timeouts": 0,
    "Requests http status 200": 10,
    "Requests total": 10
}
2024-07-17 11:15:35 301643 Generator  INFO    : Execution time: 0.067013 seconds

open your browser and go to `http://dashboards.opensiem`_ to see the generated data in the opensearch dashboards.