Running Oracle/RHEL Linux as Kubernetes Worker Nodes on AWS EKS cluster

Published Jul 1, 2020 (Updated Feb 4, 2021) by Girish Talekar • Pinned ⭐
#AMI #Worker node #EKS

Problem Statement

When working with one of my client I have been given the requirement to run RHEL as worker node on AWS EKS cluster. As I am new to this I searched on the internet for some articles, i found some articles which is good but not that what i wanted, i started reading and trying few approaches and after spending some time i was able to run RHEL as worker on EKS cluster, so just wanted to share my finding through this blog post.

There are few things you need to know below you start.

When we take an EC2 instance with any one of the base image ami like Ubuntu, Amazon Linux etc it doesn’t come with Kubernetes installed on it, to make that work as a worker node for deploying your resources, you need to install Docker, Kubernetes awscli etc. you can’t directly use any available ami from the list and use as a worker node in EKS cluster.

There are some available ami’s which has already setup form aws side below is the example, this ami’s are with ubuntu as a base image, but if you have requirement of different base OS like I have i.e. RHEL then you have to create your own custom ami by installing each component.

ami-040e5afd1b110a399
ami-0f498e4613818f406

Below are the steps we need to follow :

1. Select OL7.6 instance from aws console as a base image

   

2. Create an IAM user/role with Route53, EC2, IAM and S3 full access and attached that role to the running instance

3. Install the dependencies

    yum install -y git
    yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
    yum install -y python-pip
    pip install --upgrade awscli
    pip install --upgrade aws-cfn-bootstrap
    mkdir -p /opt/aws/bin
    ln -s /usr/bin/cfn-signal /opt/aws/bin/cfn-signal
    yum install -y http://mirror.centos.org/centos/7/extras/x86_64/Packages/container-selinux-2.107-3.el7.noarch.rpm ****can be replaced with the version required by docker****
    sed -i 's/enforcing/permissive/g' /etc/selinux/config ****If not set to permissive, the docker containers will not be able to provision and throw Permission Denied Error**
   

4. Clone the git repo and modify the install-worker.sh script for access key and region and then execute it

    git clone https://github.com/girishg4t/aws-eks-rhel-workers
    cd aws-eks-rhel-workers
    sh install-worker.sh
   

5. Go to EC2 Console and create an AMI of this server.

Create the cluster using cli

• To create the cluster through cli we need below tools from aws cli

  Installing the AWS CLI version 2

  one’s aws cli is installed you need to configure that by command

aws configure

• To use EKS for creating the cluster through cli, follow the steps in the article

  Getting started with eksctl

• one’s the image is ready copy the AMI ID from aws console like below in the config file

  

    # test-config.yaml
    apiVersion: eksctl.io/v1alpha5
    kind: ClusterConfig
  
    metadata: null
        name: test-cluster
        region: ap-south-1
        version: '1.15'
          
    nodeGroups:
      - name: ng-11
        instanceType: t2.micro
        ami: ami-xxxxxx
        desiredCapacity: 1
        ssh: null
        allow: true
  

  create the cluster by using the below command with eks cli

    eksctl create cluster -f test-config.yaml --timeout 60m
  

  wait for few minutes for cluster being ready then execute the below command to see the nodes in the cluster

    kubectl get nodes
  

Note: if you get this error message after waiting for some time like 1 hour then this is mostly because you AMI image is not setup correctly  “Error: timed out (after 1h0m0s) waiting for at least 1 nodes to join the cluster and become ready in “application-node-group”

References:

https://www.powerupcloud.com/running-red-hat-enterprise-linux-as-kubernetes-worker-nodes-xi/