Docker and Netdata – Awesome for Monitoring our home servers !

I don’t belong to any DevOps space but I keep hearing things like Docker / kubernetes and what not. I Quickly wanted to see what these are capable of and if I can use them to my advantage so that later I can see the use case for networking.

Docker so far seems to be far more capable and am enjoying it. Thanks to one of my friends who suggested this wonderful Repo, monitoring my server has been more granular.

I will have a detailed post on Juniper MX image via docker but for now, I have used it for something out of networking space to serve a small purpose.

https://github.com/netdata/netdata

This what Netdata UI looks like while monitoring my server, this is really wonderful as for the long time  I was trying to implement many Monitoring systems and most of them required some dedicated hardware or at least a Vmware spin-off instance, which is fine for me I suppose but I wanted something small and yet effective.

Netdata operates on port 19999 of localhost and should be reachable via any web-browser.

I made a small cronjob which starts this instance on every reboot.

Installation is pretty straight forward, you have to install docker and follow the below guide

https://github.com/netdata/netdata/tree/master/packaging/docker#install-netdata-with-docker

Note: For some reason, I was not able to install netdata using default installation process and Irony is that it only worked through docker which I wanted to explore.

 

-Rakesh

Advertisements

My Ignorance for a Year and Dell IDRAC

Hi,

I have a Dell R810 and it makes a lot of noise. So, I have put it somewhere remote in the home where it’s completely inhabitable for human beings. Now, the problem was always to go there and manually power-up the system.  This has been the scenario for years. So, technically if am away from my home I need to take help from my wife. All this is going well and I was always wondered should it not have a better way to do things.

Was talking to my friend and he had some paid solution for the same thing, he doesn’t even remember and that’s for his cisco gear, now for Cisco gear as long you supply power they will be powered (if the Power button is always on), for servers however just like our Personal CPU’s and Laptops you have to manually press the power button.

Then yesterday all of a sudden I took this somewhat seriously and explored options, then I understood it was sitting right inside the server, Dell has something called Integrated DRAC system which helps you do this thing, all you need to have is a proper Lan connection, the message was always there during the boot but it was ignorance in servers which made me shoo away with the boot messages. The below image is slightly dull as I have said the place where this sit’s is not great with light.

 

https://www.dell.com/support/contents/ie/en/iebsdt1/article/product-support/self-support-knowledgebase/enterprise-resource-center/systemsmanagement/idrac

After quick setup of IDRAC, I was looking for this to work over a script and below is the script

Git – https://github.com/yukthr/auts/blob/master/random_programs/server_idrac.py

And it looks something like this

 

All of this may look serious programming for someone who is not into this yet, this is really easy and most of it is nothing to do with programming, few lines from here and there and StackOverflow will do ;), again I can write this with my programming knowledge imagine what you can do with this.

 

-Rakesh

 

 

 

 

 

 

Multithreading – Whats in it ?

Hi,

I never took threading seriously, partly because I was barely getting started with programming let alone running and improving a Programs performance. I have all the time in the universe to wait for the program to run instead of making it run much faster 😉

Problem statement: Has 3 sites to ping, make 3 parallel pings (like opening 3 terminal and issuing a parallel ping statement) instead of one by one approach and has to be done with python

Threading Module comes to Rescue

https://github.com/yukthr/auts/blob/master/random_programs/threading_program.py

 

Basically, in the below output if you see two blocks, one block gets executed sequentially one ping after the other while the other one includes threading and everything is taken parallel and hence we see the output differences as well. I think am yet to figure on calculating time aspect of the threading but it is definitely faster.

Again not going much into GIL or (IO / CPU threading differences), all I can say is we can use this to enhance some day to day activities.

 

 

This may sound a little strange, macosx was not giving me any results for threading while any Linux / Unix (raspberry pi in this case) was good enough for testing the threading.

 

-Rakesh

Listing TOP 5 Processes – Top command

Continuing some exploration of Pandas, I realized in networking we often has to deal with Toptalkers, I dont have any Networking Realted Top Talker IP Data as such but wanted to see if this can checked on my Laptop’s current Processes comsuming CPU and Top processes which are repeated often.

Without dragging the topic

-> Took the text file it was delimited with space (TOP Command will generally Delimit)

-> Converted to Pandas Read FWF and then converted the file to CSV

-> Used CSV to read into specific %CPU coloumn and implemented SORT function in descending order.

-> Finally Took Counter from Collections Module and implemented it on the list.

output looksl something like this

Pandas are effective and easy, I will continue to explore some functions so as to build some scripts in day to day activities.

 

-Rakesh

 

Python Pandas – Reading text files got a lot better.

Problem Statement – Have a list of VMWare Instances and wanted to quickly iterate and see what VMs are powered on, this has nothing to do with Vmware or ESXI, look at the below image, programmatically I want to pick two specific rows and perform dictionary operations on them.

Git – https://github.com/yukthr/auts/blob/master/random_programs/pandas_vmware_esxi.py

for someone like me who is partly into programming and mostly into networking, anything effective is easy, I could have gone with the other way of doing this but  since Pandas are effective i would text parsing with them.

 

I only want two sections which are of interest to me, in a normal way maybe I should have pasted this in excel and Do a text to the column and do a manual data extraction, but wanted to do it with Pandas and they are powerful.

First things first

• Read with Pandas
• Read the text
• If possible convert into CSV (as I have some experience with csv file operations)
• Then split the Columns and make the dictionary for further use

 

 

Let’s see how this proceeds

Pandas make it so powerful to just pick Columns out of the fly, in this case, I wanted them from [1] and [6], with some code I could write up a sample script which helps in getting the power state. The next operations would be to read the os.system output and parse it but we all know how that can be done in general python use cases.

Final program looks something like this

 

-Rakesh

 

 

Working with distance sensor – solving overhead water tank problem

This is not a networking post.

Schematic , sensor code and spec  – https://www.linuxnorth.org/raspi-sump

My code – https://github.com/yukthr/auts/blob/master/random_programs/water_sensor.py

1x Breadboard

1x Raspberry pi zero w

1xhcsr04 ultrasonic sensor

2x1kohm resistors

 

Just as a side note i do not have any intro into resistors nor electronics, but what all i did was to follow some posts written by people who already did it, its not hard believe me, if i could do it any one should easily be able to do it as am very far away from electronics and programming, so let these things not overwhelm you.

 

Problem – Am not sure in other parts of the world, but place I live has an over head water Tank which stores water. So every day you technically turn on a water motor which sucks water from a reserve under the ground and pumps it to all the the way to a three store high building

So what’s the issue – The issue is that we have no clue what’s the current water level in the tank nor how long would it take to fill the water tank. There are two tribal ways by which we are addressing the problem

1- go to three stores high and sit beside the tank till it gets filled

2- let it overflow and we will know

The post aims to give an idea on a problem am no close to a professional in any of electronics nor coding but I can make it work for my self

Example of a water tank

 

So, using a raspberry pi zero , a ultrasonic sensor and telegram app did solve the issue, obviously this requires wooden finish and water proofing which is currently taken care by a plastic container.

Raspberry pie zero takes unbelievably low power and even during power outages single power bank could charge it for 40 odd hours

 

Final output – Sends a message directly to Telegram BOT

 

-Rakesh

Sonoff Flashing and steps towards Smart home

This post is no way related to Networking ! 😉

It’s always my fascination to implement a specific tech towards home automation so as to have more time in hands. The first step towards this was to use a Tp-link smart switch which came with Alexa, now problem with that is that Tplink app was not properly responding to Alexa and TP-Link smart switch in itself was costly.

I wanted something low-cost, something which I can experiment yet be cost effective, After some re-search I came across Sonoff-Basic model.

https://sonoff.itead.cc/en/products/sonoff/sonoff-basic

I integrated with Amazon echo but again there were app issues (default app is ewelink), though it was working fine something was missing, I couldn’t tinker it to my wish

I found then there is a open-source version of a similar firmware called TASMOTA and we have to flash this sonoff with the new firm-ware of Tasmota, I have to tell you here there is every possibility you would brick your device and make it useless in the process.

https://github.com/arendst/Sonoff-Tasmota/wiki/Prerequisite

Everything is so well documented, there is no need to re-iterate things here, but just to show case another successful implementation.

Need-less to say you are playing with AC/DC electrical power so take all the precautions as necessary and follow the blog articles in official tasmota page, this article does no job in listing out any implementations of flashing the device

Requirements – Though not listed in scree-shots

1- basic solder guns

2-solder pins

3.Raspberry pi (you can also do it with Serial USB but i used Rasepberry pi)

 

 

1. Open the Sonoff basic cover

2. Connect these to sonoff switch and Raspberry pi as instructed in the wiki

This is how connection would look like

Connect to Raspberry pi

Flashing Sonoff , this is the tricky part, if possible get a second pair of hands.

New firmware would take off, Sonoff would boot in an AP mode so the wireless SSID SONOFF_X will be seen in your wireless connections, connect to it and you should be able to configure the device and configure the wireless parameters

 

You can configure MQTT topic subscription or just ask sonoff to emulate Wemo switch so that it can be used with a Smart home skill like YETI in Amazon echo (alexa)

I have cut a spike cable, connected sonoff. Spike connects to entertainment system which can be programmed to switchoff after a particular time in night or invoke it with voice commands from echo.

This is some start towards affordable home automation, I will post more about PIR sensors which i have been using and other details as time permits. Hass.io is looking promising and should be a good start as well.

-Rakesh