Linux / Open Source Forum

Monday, November 11, 2019

KIND - Kubernetes IN Docker

We can easily up and running K8S cluster for testing, This very lightweight as compared to other local setups.

This post is a reference for those who are trying to install the Kubernetes cluster on your ubuntu machine.

My Ubuntu 18.04 TLS server is a VirtualBox VM and I have installed the following dependencies on that server.

1. Install Docker

sudo apt-get update
sudo apt-get install apt-transport-https ca-certificates curl gnupg-agent software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo apt-key fingerprint 0EBFCD88
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io

2. Install GO Language
https://golang.org/dl/

$ wget https://dl.google.com/go/go1.13.4.linux-amd64.tar.gz
$sudo tar -C /usr/local -xzf go1.13.4.linux-amd64.tar.gz
$ export PATH=$PATH:/usr/local/go/bin
$ go version
go version go1.13.4 linux/amd64

3. kubectl

$ curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 44.5M 100 44.5M 0 0 1209k 0 0:00:37 0:00:37 --:--:-- 1182k
$ chmod +x ./kubectl
$ sudo mv ./kubectl /usr/local/bin/kubectl

Now install KIND

https://github.com/kubernetes-sigs/kind

$ GO111MODULE="on" go get sigs.k8s.io/kind@v0.5.1
go: finding sigs.k8s.io v0.5.1
$ rm go1.13.4.linux-amd64.tar.gz
$ export PATH=$PATH:/home/ajeesh/go/bin
$ kind version
v0.5.1
$ kind create cluster
Creating cluster "kind" ...
✓ Ensuring node image (kindest/node:v1.15.3) 🖼
✓ Preparing nodes 📦
✓ Creating kubeadm config 📜
✓ Starting control-plane 🕹️
✓ Installing CNI 🔌
✓ Installing StorageClass 💾
Cluster creation complete. You can now use the cluster with:
export KUBECONFIG="$(kind get kubeconfig-path --name="kind")"
kubectl cluster-info

$ export KUBECONFIG="$(kind get kubeconfig-path --name="kind")"
$ kind get kubeconfig-path
/home/ajeesh/.kube/kind-config-kind

$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b52ee9180210 kindest/node:v1.15.3 "/usr/local/bin/entr…" 14 minutes ago Up 13 minutes 35507/tcp, 127.0.0.1:35507->6443/tcp kind-control-plane

$ kubectl get nodes

NAME STATUS ROLES AGE VERSION

kind-control-plane Ready master 17m v1.15.3

But this is a single node cluster. But if you want to create a multi-node HA cluster we need to do the following settings.

First, delete the current cluster.

$ kind delete cluster
Deleting cluster "kind" ...
$KUBECONFIG is still set to use /home/ajeesh/.kube/kind-config-kind even though that file has been deleted, remember to unset it
$ unset KUBECONFIG
This will delete the kubeconfig file on your .kube folder.
/.kube$ ls
cache http-cache

$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

A cluster with 3 control-plane nodes and 3 workers

$ cat multi-node-kind.yml
kind: Cluster
apiVersion: kind.sigs.k8s.io/v1alpha3
nodes:
- role: control-plane
- role: control-plane
- role: control-plane
- role: worker
- role: worker
- role: worker
~$ kind create cluster --config multi-node-kind.yml
Creating cluster "kind" ...
✓ Ensuring node image (kindest/node:v1.15.3) 🖼
✓ Preparing nodes 📦📦📦📦📦
✓ Configuring the external load balancer ⚖️
✓ Creating kubeadm config 📜
✓ Starting control-plane 🕹️
✓ Installing CNI 🔌
✓ Installing StorageClass 💾
✓ Joining more control-plane nodes 🎮
✓ Joining worker nodes 🚜
Cluster creation complete. You can now use the cluster with:
export KUBECONFIG="$(kind get kubeconfig-path --name="kind")"
kubectl cluster-info

I have some issue with resources so I have reduced worker node to 1 from 3

$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
kind-control-plane Ready master 2m20s v1.15.3
kind-control-plane2 Ready master 106s v1.15.3
kind-worker Ready 46s v1.15.3
:~$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a0e52dd3effa kindest/node:v1.15.3 "/usr/local/bin/entr…" 4 minutes ago Up 3 minutes kind-worker
2a6c8833c3cb kindest/node:v1.15.3 "/usr/local/bin/entr…" 4 minutes ago Up 3 minutes 35213/tcp, 127.0.0.1:35213->6443/tcp kind-control-plane2
4e38366ad4a7 kindest/haproxy:2.0.0-alpine "/docker-entrypoint.…" 4 minutes ago Up 4 minutes 37331/tcp, 127.0.0.1:37331->6443/tcp kind-external-load-balancer
8a0ce1959722 kindest/node:v1.15.3 "/usr/local/bin/entr…" 4 minutes ago Up 3 minutes 36145/tcp, 127.0.0.1:36145->6443/tcp kind-control-plane

:~$ kind get nodes
kind-worker
kind-control-plane2
kind-external-load-balancer
kind-control-plane
~$ kubectl -n kube-system get all
NAME READY STATUS RESTARTS AGE
pod/coredns-5c98db65d4-hmrxc 1/1 Running 0 4m31s
pod/coredns-5c98db65d4-vgj9w 1/1 Running 0 4m31s
pod/etcd-kind-control-plane 1/1 Running 0 3m36s
pod/etcd-kind-control-plane2 1/1 Running 0 4m12s
pod/kindnet-7754r 1/1 Running 1 4m13s
pod/kindnet-9c4rt 1/1 Running 1 4m31s
pod/kindnet-b2td4 1/1 Running 1 3m13s
pod/kube-apiserver-kind-control-plane 1/1 Running 0 3m36s
pod/kube-apiserver-kind-control-plane2 1/1 Running 0 4m12s
pod/kube-controller-manager-kind-control-plane 1/1 Running 1 3m58s
pod/kube-controller-manager-kind-control-plane2 1/1 Running 0 3m58s
pod/kube-proxy-c628w 1/1 Running 0 4m13s
pod/kube-proxy-p9787 1/1 Running 0 4m31s
pod/kube-proxy-zf6pm 1/1 Running 0 3m13s
pod/kube-scheduler-kind-control-plane 1/1 Running 1 3m58s
pod/kube-scheduler-kind-control-plane2 1/1 Running 0 4m12s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kube-dns ClusterIP 10.96.0.10 53/UDP,53/TCP,9153/TCP 4m46s
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/kindnet 3 3 3 3 3 4m43s
daemonset.apps/kube-proxy 3 3 3 3 3 beta.kubernetes.io/os=linux 4m45s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/coredns 2/2 2 2 4m46s
NAME DESIRED CURRENT READY AGE
replicaset.apps/coredns-5c98db65d4 2 2 2 4m31s

Tuesday, October 8, 2019

VirtualBox 6 upgrade and issues for the latest version

Upgrade your VirtualBox to the latest release 6

Here I am showing how i have upgraded/downgrade my VirtualBox on Ubuntu machine.
NB: You need to backup your virtual servers before you perform the steps.

Please find the below steps where i have followed for upgrading my virtualbox version 5.1.38 to latest stable version 6.0.12. I didn't take any backup :)

ajeesh@ajeesh-Aspire-A515-51G:~$ ps aux | grep virt
ajeesh   19129 0.1 0.1 167452 12920 ?        S    21:52   0:00 /usr/lib/virtualbox/VBoxXPCOMIPCD
ajeesh   19134 0.4 0.2 761312 23784 ?        Sl   21:52   0:01 /usr/lib/virtualbox/VBoxSVC --auto-shutdown

ajeesh@ajeesh-Aspire-A515-51G:~$ kill -9 19129 19134

root@ajeesh-Aspire-A515-51G:/etc/apt/sources.list.d# apt-get install virtualbox-6.0
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
libsdl-ttf2.0-0
The following packages will be REMOVED:
virtualbox virtualbox-ext-pack virtualbox-qt
The following NEW packages will be installed:
libsdl-ttf2.0-0 virtualbox-6.0
0 upgraded, 2 newly installed, 3 to remove and 258 not upgraded.
Need to get 109 MB of archives.
After this operation, 147 MB of additional disk space will be used.
Do you want to continue? [Y/n] y

Setting up virtualbox-6.0 (6.0.12-133076~Ubuntu~xenial) ...
addgroup: The group `vboxusers' already exists as a system group. Exiting.
Processing triggers for libc-bin (2.23-0ubuntu10) ...
root@ajeesh-Aspire-A515-51G:

But this version 6.0.12 is causing some issue for my Vagrant containers.

vagrant-container$ vagrant up

The provider 'virtualbox' that was requested to back the machine
'kmaster' is reporting that it isn't usable on this system. The
reason is shown below:

Vagrant has detected that you have a version of VirtualBox installed
that is not supported by this version of Vagrant. Please install one of
the supported versions listed below to use Vagrant:

4.0, 4.1, 4.2, 4.3, 5.0, 5.1

A Vagrant update may also be available that adds support for the version
you specified. Please check www.vagrantup.com/downloads.html to download
the latest version.
ajeesh@ajeesh-Aspire-A515-51G:

So i have downgraded the VirtualBox version to 5.1.38

sources.list.d# apt-get install virtualbox-5.1
Reading package lists... Done
Building dependency tree
Reading state information... Done
Do you want to continue? [Y/n] y
Get:1 http://download.virtualbox.org/virtualbox/debian xenial/contrib amd64 virtualbox-5.1 amd64 5.1.38-122592~Ubuntu~xenial [66.0 MB]
Fetched 66.0 MB in 57s (1,147 kB/s)
Preconfiguring packages ...

Setting up virtualbox-5.1 (5.1.38-122592~Ubuntu~xenial) ...
addgroup: The group `vboxusers' already exists as a system group. Exiting.
root@ajeesh-Aspire-A515-51G:

vagrant-container$ kubectl get nodes
NAME                   STATUS    ROLES     AGE       VERSION
kmaster.example.com    Ready     master    2d21h     v1.16.1
kworker1.example.com   Ready         2d21h     v1.16.1
kworker2.example.com   Ready         2d21h     v1.16.1

Thursday, October 3, 2019

LND Lighning Network Vulnerability reported

LND Lighning Network Vulnerability reported

Recently lightning network developer Russel updated a serious vulnerability for the old versions. The versions which is less than v0.7.0

    CVE-2019-12998 c-lightning < 0.7.1
    CVE-2019-12999 lnd < 0.7
    CVE-2019-13000 eclair <= 0.3

The issue he described as below:

A lightning node accepting a channel must check that the funding transaction
output does indeed open the channel proposed. Otherwise an attacker can claim
to open a channel but either not pay to the peer, or not pay the full amount.
Once that transaction reaches the minimum depth, it can spend funds from the
channel. The victim will only notice when it tries to close the channel and none
of the commitment or mutual close transactions it has are valid.

Solution
--------

Once the funding transaction is seen, peers MUST check that the outpoint as
described in `funding_created`[1] is a funding transaction output[2] with
the amount described in `open_channel`[3].

Fixed versions:
c-lightning: v0.7.1 and above
lnd: v0.7.1 and above
eclair: v0.3.1 and above

So the best way to fix the issue is you need t upgrade to the latest release, While right this i can see the latest version for the lnd is v0.8.0-Beta. From this release onwards, lnd will only support database upgrades from the previous major release. So that means those who are running on v0.6.0 would be required to upgrade v0.7.0 first and then to v0.8.0.

VMWare Tools : Not Running

In VMWare vSphere Client you will see the following errors.

VMWare Tools : Not Running ( Not Installed)

We can easily fix this issue by installing open-vm-tools package in linux machine.

root@testing:/home/ubuntu# apt-get install open-vm-tools

Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
libdumbnet1 libicu52 zerofree
Suggested packages:
open-vm-tools-desktop
The following NEW packages will be installed:
libdumbnet1 libicu52 open-vm-tools zerofree
0 upgraded, 4 newly installed, 0 to remove and 218 not upgraded.
Need to get 7,237 kB of archives.
After this operation, 30.8 MB of additional disk space will be used.
Do you want to continue? [Y/n] y

After installing this package you can see your IP address and MAC address using this vmware tool.

Saturday, September 28, 2019

upgrade Visual Studio Code version in Ubuntu

It is very easy for upgrading your Visual Studio Code in your Ubuntu desktop machine.

ajeesh@ajeesh-Aspire-A515-51G:~$ code --version
1.35.1
c7d83e57cd18xxx2843bda1bcf21f
x64

root@ajeesh-Aspire-A515-51G:/home/ajeesh# apt-get install code
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be upgraded:
code
1 upgraded, 0 newly installed, 0 to remove and 265 not upgraded.
Need to get 55.4 MB of archives.
After this operation, 45.8 MB of additional disk space will be used.
Get:1 https://packages.microsoft.com/repos/vscode stable/main amd64 code amd64 1.38.1-1568209190 [55.4 MB]
Fetched 55.4 MB in 23min 10s (39.9 kB/s)
(Reading database ... 379957 files and directories currently installed.)
Preparing to unpack .../code_1.38.1-1568209190_amd64.deb ...
Unpacking code (1.38.1-1568209190) over (1.35.1-1560350270) ...
Processing triggers for gnome-menus (3.13.3-6ubuntu3.1) ...
Processing triggers for desktop-file-utils (0.22-1ubuntu5.1) ...
Processing triggers for bamfdaemon (0.5.3~bzr0+16.04.20160824-0ubuntu1) ...
Rebuilding /usr/share/applications/bamf-2.index...
Processing triggers for mime-support (3.59ubuntu1) ...
Setting up code (1.38.1-1568209190) ... root@ajeesh-Aspire-A515-51G:/home/ajeesh# exit

ajeesh@ajeesh-Aspire-A515-51G:~$ code --version
1.38.1
b37e54c98e1xxe5a3761284e3ffb0
x64
ajeesh@ajeesh-Aspire-A515-51G:~$

Thursday, February 28, 2019

How can we reduce the docker image size

DIVE

This tool provides a way to discover and explore the contents of a docker image. Additionally the tool estimates
the amount of wasted space and identifies the offending files from the image.

root@ajeesh-desktop:/home/ajeesh# wget https://github.com/wagoodman/dive/releases/download/v0.6.0/dive_0.6.0_linux_amd64.deb

root@ajeesh-desktop:/home/ajeesh# apt install ./dive_0.6.0_linux_amd64.deb

root@ajeesh-desktop:/home/ajeesh# dive test_frontend:v1
Fetching image...
Parsing image...
├─ [layer: 1] 02c8cd0778ef78e : [==============================>] 100 % (29046/29046)
├─ [layer: 2] 0e24a3d04ebc058 : [==============================>] 100 % (4149/4149)
├─ [layer: 3] 1fbf962f1faa79c : [==============================>] 100 % (268/268)
├─ [layer: 4] 263da0d188f39bc : [==============================>] 100 % (20812/20812)
├─ [layer: 5] 2c6d59c15b5a74f : [==============================>] 100 % (22112/22112)
├─ [layer: 6] 4a3e40ad2625f55 : [==============================>] 100 % (6/6)
├─ [layer: 7] 5f67701353f1632 : [==============================>] 100 % (23/23)
├─ [layer: 8] 6e9ac776b0f4ad8 : [==============================>] 100 % (2/2)
├─ [layer: 9] 71f81c9582dbf66 : [==============================>] 100 % (25/25)
├─ [layer: 10] 84a49a835b59732 : [==============================>] 100 % (2/2)
├─ [layer: 11] 9a9aa5f37fdc24c : [==============================>] 100 % (9285/9285)
├─ [layer: 12] bb8588e06b91e7b : [==============================>] 100 % (3/3)
├─ [layer: 13] cacd4f0f725d2c5 : [==============================>] 100 % (4/4)
├─ [layer: 14] d107e0074bfc596 : [==============================>] 100 % (7010/7010)
├─ [layer: 15] d7ecfda03fa2ea7 : [==============================>] 100 % (6293/6293)
├─ [layer: 16] f08e6b62ac1a1d3 : [==============================>] 100 % (1354/1354)
├─ [layer: 17] ff6cffce858c4c9 : [==============================>] 100 % (1428/1428)
├─ [layer: 18] ffa2599ac9ec53e : [==============================>] 100 % (885/885)
╧
Analyzing image...
Building cache...

Tuesday, April 12, 2016

Export a Linux virtual machine from VMWare EXSI server and Import to AWS

We can export our EXSI virtual machine images to aws and create the exact same machine as your are running on your EXSI machine.

For this process we need to do the following steps.

1. Download and install AWS CLI
2. Configure the access key with your aws console (IAM access)
3. Export the virtual machine from EXSI machine to your local machine as ova formate
4. Import the OVA image file to your aws S3 bucket

Setup AWS CLI

Download the AWS CLI here:
https://s3.amazonaws.com/aws-cli/AWSCLI64.msi
After that complete the installation.

[root@ip-172-31-56-125 ec2-user]# aws configure
AWS Access Key ID [None]: AKIAXXXXXXXWHTA
AWS Secret Access Key [None]: svcngBGXXXnijOWPLDXXXXXXXHaqs
Default region name [None]: us-east-1
Default output format [None]:
[root@ip-172-31-56-125 ec2-user]#

After the configuration we can test and confirm everything seems to be working fine.

[root@ip-172-31-56-125 ec2-user]# aws ec2 create-security-group --group-name AJVMGRP --description "testing vmtestimptest"
{
    "GroupId": "sg-2a250452"
}
[root@ip-172-31-56-125 ec2-user]#

So we have successfully configured the AWS CLI on this machine

[root@ip-172-31-56-125 ec2-user]# aws ec2 import-image --cli-input-json "{ \"Description\": \"Kalilinux-2016\", \"DiskContainers\": [ {\"Description\": \"Kalilinux-2016\", \"UserBucket\": { \"S3Bucket\": \"vmimportajeesh\", \"S3Key\" : \"CentOS-6.4-i386-minimal.ova\" } } ]}"

{
    "Status": "active",
    "Description": "Kalilinux-2016",
    "Progress": "2",
    "SnapshotDetails": [
        {
            "UserBucket": {
                "S3Bucket": "vmimportajeesh",
                "S3Key": "CentOS-6.4-i386-minimal.ova"
            },
            "DiskImageSize": 0.0
        }
    ],
    "StatusMessage": "pending",
    "ImportTaskId": "import-ami-fggrwl9r"
}
[root@ip-172-31-56-125 ec2-user]#

On Our EXSI Box
We need to create a vm. The VM should be capable of accepting DHCP assign IP address and remove vmware-tool if we installed any.

Export VM in OVA format

File > Export > Export OVF Template [Single format]

Create a S3 bucket

Create a S3 bucket and make note the ARN

Import the OVA file to S3 Bucket

You can upload the particular OVA file to your S3 bucket

We need to change the service name to vmie.amazoneaws.com(Trust Relationship) and add the following condition

Create a new role[eg: vmimport] and assign administrative privilege

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "vmie.amazonaws.com"
},
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"sts:ExternalId": "vmimport"
}
}
}
]
}

Import the OVA to AWS

[root@ip-172-31-56-125 ec2-user]# aws ec2 import-image --cli-input-json "{ \"Description\": \"Kalilinux-2016\", \"DiskContainers\": [ {\"Description\": \"Kalilinux-2016\", \"UserBucket\": { \"S3Bucket\": \"vmimportajeesh\", \"S3Key\" : \"CentOS-6.4-i386-minimal.ova\" } } ]}"
-----
{
    "Status": "active",
    "Description": "Kalilinux-2016",
    "Progress": "2",
    "SnapshotDetails": [
        {
            "UserBucket": {
                "S3Bucket": "vmimportajeesh",
                "S3Key": "CentOS-6.4-i386-minimal.ova"
            },
            "DiskImageSize": 0.0
        }
    ],
    "StatusMessage": "pending",
    "ImportTaskId": "import-ami-fggrwl9r"
}
---

We will get a image ID [ import-ami-fggrwl9r ]

to check the status::
aws ec2 describe-import-image-tasks --image-task-ids “import-ami-fggrwl9r”

[root@ip-172-31-56-125 ec2-user]# aws ec2 describe-import-image-tasks --import-task-ids "import-ami-fggrwl9r"
{
    "ImportImageTasks": [
        {
            "Status": "active",
            "Description": "Kalilinux-2016",
            "Progress": "28",
            "SnapshotDetails": [
                {
                    "UserBucket": {
                        "S3Bucket": "vmimportajeesh",
                        "S3Key": "CentOS-6.4-i386-minimal.ova"
                    },
                    "DiskImageSize": 346110976.0,
                    "Format": "VMDK"
                }
            ],
            "StatusMessage": "converting",
            "ImportTaskId": "import-ami-fggrwl9r"
        }
    ]
}
[root@ip-172-31-56-125 ec2-user]#

then this will change to

   "StatusMessage": "updating",
   "ImportTaskId": "import-ami-fggrwl9r"

   "StatusMessage": "booting",
   "ImportTaskId": "import-ami-fggrwl9r"

"StatusMessage": "preparing ami",
"ImportTaskId": "import-ami-fggrwl9r"

[root@ip-172-31-56-125 ec2-user]# aws ec2 describe-import-image-tasks --import-task-ids "import-ami-fggrwl9r"
{
    "ImportImageTasks": [
        {
            "Status": "completed",
            "LicenseType": "BYOL",
            "Description": "Kalilinux-2016",
            "ImageId": "ami-4e6e7924",
            "Platform": "Linux",
            "Architecture": "x86_64",
            "SnapshotDetails": [
                {
                    "UserBucket": {
                        "S3Bucket": "vmimportajeesh",
                        "S3Key": "CentOS-6.4-i386-minimal.ova"
                    },
                    "SnapshotId": "snap-649ee37d",
                    "DiskImageSize": 346110976.0,
                    "DeviceName": "/dev/sda1",
                    "Format": "VMDK"
                }
            ],
            "ImportTaskId": "import-ami-fggrwl9r"
        }
    ]
}
Finally this image is available on your AWS console. Using this you can launch a new vps.