• About WordPress
    • WordPress.org
    • Documentation
    • Support
    • Feedback
  • Log In
  • Register
  • Home
  • About Us
  • Blog
  • Courses
  • Contact Us

Have any question?

101daysofdevops@gmail.com
RegisterLogin
101DaysofDevops
  • Home
  • About Us
  • Blog
  • Courses
  • Contact Us

Blog

  • Home
  • Blog
  • Blog
  • Kubewatch — A watcher for Kubernetes

Kubewatch — A watcher for Kubernetes

  • Posted by lakhera2020
  • Date February 7, 2021
  • Comments 3 comments

As per the official github link

kubewatch is a Kubernetes watcher that publishes notifications

to Slack/hipchat/mattermost/flock channels. It watches the cluster

for resource changes and notifies them through webhooks.

Basically kubewatch looks for events like pod/deployment/confimap creation/deletion and sent notification to selected channels like

 – slack

 – hipchat

 – mattermost

 – flock

 – webhook

 – smtp

Installation

In this blog, we will use helm to install kubewatch and use slack as the notification medium. But before setting up helm and slack, you must need to have your running Kubernetes cluster. Please check the link on how to configure the Kubernetes cluster using kind.

Configuring Slack

  • Go to your slack workspace, and you have the option to create a new channel, or you can use an existing channel. For this demo, I am going to use an existing channel slacktest(but creating a new channel is pretty easy, you need to click on Create a new channel).
  • Once you have a slack channel, the next step is to get a slack token to integrate with Kubewatch. To create a slack token, go to https://api.slack.com/apps/new and create a new app. Give your App Name, for eg: kubewatch, and select your Development Slack Workspace for eg.100daysofdevops in this case.
  • Now scroll down and click on Oauth & Permissions and note Bot User OAuth Access token.

  • One more step you need to perform is to invite the Bot to the channel

/invite @BOT_NAME

/invite @kubewatch

Using helm

Once you have helm installed in your cluster(if you want more information about helm and install it, please follow this link). 

  • Add the bitnami repo.
helm repo add bitnami https://charts.bitnami.com/bitnami
"bitnami" has been added to your repositories
  • To verify it
helm repo list                                          
NAME URL
bitnami https://charts.bitnami.com/bitnami
  • Next, we will run a repo update to make sure all the charts are updated and in sync.
helm repo update
Update Complete. ⎈Happy Helming!⎈
  • Now search for kubewatch chart
helm search repo kubewatch                  
NAME CHART VERSION APP VERSION DESCRIPTION
bitnami/kubewatch 3.1.0 0.1.0 Kubewatch is a Kubernetes watcher that currentl...
  • As we know all helm chart comes with a default set of values but in this case, we want to modify these values according to our requirement.
helm show values bitnami/kubewatch > ~/kubewatch.yaml
  • Now open this file and modify few parameters. First, let start with slack and add the channel to send notification and add token we have created during the slack section.
slack:
enabled: true
# Slack channel to notify
channel: "slacktest"
# Slack bots token. Create using: https://my.slack.com/services/new/bot
# and invite the bot to your channel using: /join @botname
token: "xoxb-XXXXXXX"
  • In the next step, we need to define what I want to monitor. So, in this case, I want to monitor only the events from the default namespace(namespaceToWatch: “default”) and the resources to watch(deployment, services, pod). If you want to monitor any other resources, changes the parameter to true(for eg. replicaset: false to replicaset: true).
# namespace to watch, leave it empty for watching all.
namespaceToWatch: "default"
# Resources to watch
resourcesToWatch:
deployment: true
replicationcontroller: false
replicaset: false
daemonset: false
services: true
pod: true
job: false
persistentvolume: false
  • One important parameter you need to set is rbac to true. If you set it to a default value to false, then the service account this helm chart creates doesn’t have access to list Kubernetes resources such as pod, deployments, services, etc.
rbac:
create: true
  • With all the configuration in place, its time to deploy the helm chart with our customized values
helm install my-kubewatch bitnami/kubewatch --values ~/kubewatch.yaml 
NAME: my-kubewatch
LAST DEPLOYED: Sun Feb 7 21:54:07 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
** Please be patient while the chart is being deployed **
To verify that kubewatch has started, run:
kubectl get deploy -w --namespace default my-kubewatch
  • If you now execute the command shown at the last command’s output, you will see the my-kubewatch pod created in the default namespace.
kubectl get deploy -w --namespace default my-kubewatch
NAME READY UP-TO-DATE AVAILABLE AGE
my-kubewatch 0/1 1 0 11s
my-kubewatch 1/1 1 1 16s

Testing

  • Try to create any pod in the default namespace
kubectl run nginx2 --image=nginx              
pod/nginx2 created
  • If you go to the slack channel, you will see a notification like this
  • You can also tail the logs of your pod to verify these notifications
kubectl logs my-kubewatch-5ff89fc866-lxvw8
time="2021-02-08T06:37:46Z" level=info msg="Processing add to pod: default/nginx2" pkg=kubewatch-pod
time="2021-02-08T06:37:46Z" level=info msg="Processing update to pod: default/nginx2" pkg=kubewatch-pod
time="2021-02-08T06:37:46Z" level=info msg="Processing update to pod: default/nginx2" pkg=kubewatch-pod
2021/02/08 06:37:47 Message successfully sent to channel CRPP4PCR4 at 1612766267.001300
2021/02/08 06:37:47 Message successfully sent to channel CRPP4PCR4 at 1612766267.001400
time="2021-02-08T06:37:48Z" level=info msg="Processing update to pod: default/nginx2" pkg=kubewatch-pod
2021/02/08 06:37:49 Message successfully sent to channel CRPP4PCR4 at 1612766269.001500

Wrapping Up

Kubewatch is a powerful tool and sends a notification for any events happening in your Kubernetes cluster. But please configure it wisely, as too many notifications in the busy cluster might lead to notification fatigue, and you will start missing the important ones.

Tag:devops, docker, kubernetes, kubewatch, monitoring, notification, slack, technology

  • Share:
author avatar
lakhera2020

Previous post

Introduction to Helm and Creating your first Helm Chart
February 7, 2021

Next post

Extending Kubernetes with the plugin using Krew
February 11, 2021

You may also like

Am I reading the iostat command output correctly?
25 April, 2022

Iostat command came from the same sysstat family package # rpm -qf `which iostat` sysstat-11.7.3-6.el8.x86_64 It mainly read data from /proc/diskstats # cat /proc/diskstats 259 0 nvme1n1 147 0 6536 …

Debugging Performance Issue using SAR
21 April, 2022

What is SAR? SAR is a utility used to collect and report system activity. It collects data relating to most core system functions and writes those metrics to binary data …

4 common Kubernetes Pods Error and Debugging
20 April, 2022

Why do Kubernetes Pods fail? The two most common reasons for Kubernetes pod failure is The container inside the pod doesn’t start, which we also call a startup failure. The …

    3 Comments

  1. mayuresh
    February 8, 2021
    Reply

    Hi Prashant,

    Thanks for informative blog. I am trying to follow up step but got stuck in .

    One more step you need to perform is to invite the Bot to the channel
    /invite @BOT_NAME

    /invite @kubewatch

    can you please explain more about how to do this step rest of the steps are working fine.

    Many thanks

    • lakhera2020
      February 8, 2021
      Reply

      Mayur, Can you please post the error you are getting?

  2. Natan Yellin
    January 30, 2022
    Reply

    Hey, we built a tool on top of Kubewatch that lets you enrich the notifications with extra data and take remediation actions directly from Slack. Would love to see a writeup too!

    https://github.com/robusta-dev/robusta
    http://robusta.dev/

Leave A Reply Cancel reply

Your email address will not be published. Required fields are marked *

Recent Posts

  • Am I reading the iostat command output correctly?
  • Debugging Performance Issue using SAR
  • 4 common Kubernetes Pods Error and Debugging
  • My road to Gremlin Chaos Engineering Practitioner Certificate
  • My road to Certified Kubernetes Security Specialist (CKS)

Recent Comments

  • lakhera2020 on Debugging Performance Issue using SAR
  • Anonymous on Debugging Performance Issue using SAR
  • Pety on Day 2 – MetalLB Load Balancer for Bare Metal Kubernetes
  • akashambasta on Day 1 – AWS IAM User
  • rd on 100 Days of AWS

 

101daysofdevops@gmail.com

  • Home
  • About Us
  • Courses
  • Blog

© 101daysofdevops. All rights reserved.

Login with your site account

Lost your password?

Not a member yet? Register now

Register a new account

Are you a member? Login now