Install and create a cluster of ES Servers

There are many various installation notes around the various blogs you may find. So did i, but unfortunately none were in complete state or so. The procedure stated by them might have worked out for them at that time but unfortunately didn’t work for me. So, let me share the process i followed for setting the things right in installing / setting up a cluster of ES Servers.

Perquisites

You may have a physical / virtual machine with a flavour of Ubuntu or CentOS or either one you feel comfortable with. Will recommend to go with the latest edition that will help you serve with latest updates / security patches. Also will recommend a minimum of 1GB of RAM. But you will be in a better understand the requirements of the server that suites you the best.

Our team here had gone in with 3 Virtual Machines having CentOS 7 with having 8 GB of RAM.

Step 1. Installation

Before we start installing Elastic Search servers, first we will need to install Java in order to run the same. Follow the steps for installing Java (Current version of java is 8.):

1. 1. Update the server with the following command.

      sudo apt-get update

   2. Add the Official Oracle Java repository

      sudo add-apt-repository ppa:webupd8team/java

   3. Now refresh the package list

      sudo apt-get update

   4. Now install the java using the following command:

      sudo apt-get install oracle-java8-installer

   5. Once installed, verify the installation by checking the java version.

      java -version

There are two ways of installing the server which are easy / convenient to go along with. One using Yum / APT, other is by download the required version from the server and installing it. We are going to download and install the desired version from the site. This is much convenient then the other way of installations.

1. Download the latest version from the server. Go for the RPM version of the installations, it creates all the necessary installations configurations. People who are expert enough may also go with the manual version through the zip / tar variants of the installation process. Note: At this moment, the latest version of ES was 6.1.3 – so we download and install the same

   cd ~
   wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.1.3.rpm

2. Install the downloaded package.

   sudo dpkg -i elasticsearch-6.1.3.rpm

Step 2: Configurations

Open the file

/etc/elasticsearch/elasticsearch.yml.

(This will be the path of the configuration path in case you have followed up the installation steps shared above.

1. Binding to VPN IP Address or Interface:
   • If we do not bind to any ip address in configuration, by default elastic search will be available only from localhost. At this point, the system is not accessible from other machines on the network or publicaly.
   • If we want to access it from other machines, we have to bind it to the desired IP address. Now there are times we may bind it to the internal network ip address or from external resource. In such scenario, we may point it to either IP addresses.
   • But one of the document shared a good insight. Bind it to the interfaces. Say we have the interface name set to – ‘tun0’. Other available interface is ‘local’.  Lets bind the same the server to this two interfaces.

     network.host: [_tun0_, _local_]

2. Set the name of the cluster.

   custer.name: production

3. Set the nodes name. Assuming we have node1, node2, node3 as hostnames for each hosts.
   node.name: node1
   Ps.Note: We will have to set the node name to its individual machines configuration.
4. Now set the discovery hosts. Here, we will configure the initial lists of nodes that will be contacted to discover and form a cluster. This is necessary in a unicast network.

   discovery.zen.ping.unicast.hosts: ["node1", "node2", "node3"]

5. Now this node names that we have set here, we will need to set the IP address pointing to each machine. Hence we will edit /etc/hosts file and set the ip address / hosts name on all machines.

Save and exit the elasticsearch.yml

Now start elasticsearch:

sudo systemctl start elasticsearch

Also configure the system to start Elastic Search on boot up:

sudo systemctl enable elasticsearch

Now there is one thing that is very critical / important for us to enable the cluster. We need to enable port 9300 into the firewall. (Pl. note, since we started with CentOS 7, we will go ahead configuring the same with that in the environment)

firewall-cmd --zone=public --add-port=9200/tcp --permanent
firewall-cmd --zone=public --add-port=9300/tcp --permanent
firewall-cmd --reload

Once the firewall is set to allow the machine to access port 9300 publicly, each servers should now be in state of communicating with each other to form a cluster. (Pl. Note, in above we have opened the port to be able to access the system by anyone who can connect the machine on public interface. It is recommended for you to keep the same limited to internal network / specific Ip address for security purpose.)

Now you can check the state of the cluster by issuing the following command:

curl -XGET 'http://localhost:9200/_cluster/state?pretty'

This surely should give u a clustered environment with 3 nodes connected with each other. The above should give you an output like the following:

{
  "cluster_name": "production",
  "compressed_size_in_bytes": 358,
  "version": 20,
  "state_uuid": "pTJFgszvT_qz8UJGezyJSQ",
  "master_node": "kXwy2-X7SDSqjtQZgLxgjg",
  "blocks": {},
  "nodes": {
    "lY_a_D2dT5uLM5NNCLsTTA": {
      "name": "node1",
      "ephemeral_id": "3XiogkxlTHKzpYY2m5iVug",
      "transport_address": "192.168.0.167:9300",
      "attributes": {}
    },
    "o8pHnUXaQHKLAL4f1oAiFA": {
      "name": "node2",
      "ephemeral_id": "Bjv0G8EXT0KvDyoS7WL4Qg",
      "transport_address": "192.168.0.168:9300",
      "attributes": {}
    },
    "kXwy2-X7SDSqjtQZgLxgjg": {
      "name": "node3",
      "ephemeral_id": "hExDdFoWRXyBbiexF3MzpQ",
      "transport_address": "192.168.0.169:9300",
      "attributes": {}
    }
  },
  "metadata": {
    "cluster_uuid": "A1MnDXSiRB6rF_eCh6llXQ",
    "templates": {},
    "indices": {},
    "index-graveyard": {
      "tombstones": []
    }
  },
  "routing_table": {
    "indices": {}
  },
  "routing_nodes": {
    "unassigned": [],
    "nodes": {
      "o8pHnUXaQHKLAL4f1oAiFA": [],
      "kXwy2-X7SDSqjtQZgLxgjg": [],
      "lY_a_D2dT5uLM5NNCLsTTA": []
    }
  },
  "snapshots": {
    "snapshots": []
  },
  "restore": {
    "snapshots": []
  },
  "snapshot_deletions": {
    "snapshot_deletions": []
  }
}

Another very important configuration that i came across for ES – for sake of performance and stability is to Enable Memory Locking. Following are the steps for the same.

1. Edit the ES Configuration and uncomment or add the following line:

   bootstrap.mlockall: true

   Save and exit the file/

2.  Next – open the following file for editing: /etc/sysconfig/elasticsearch and change a few settings in the same.
   • Set the Heap Size for ES. You can set it about 50% of the available memory. (Pl. note, the max heapsize allocation recommended is not more then 32GB).

     ES_HEAP_SIZE=4g

   • Uncomment the following line

     MAX_LOCKED_MEMORY=unlimited

   • Save and exit
3. Now edit the ES Systemd unit file

   • sudo vi /usr/lib/systemd/system/elasticsearch.service

   • Uncomment or add the following:

     LimitMEMLOCK=infinity

   • Save and Exit
4. Now reload the systemctl daemon and restart Elasticsearch to put in the changes into place

   sudo systemctl daemon-reload
   sudo systemctl restart elasticsearch

5. Now verify Mlockall Status. Issue the following command to check the same:

   curl http://localhost:9200/_nodes/process?pretty

   Each node should have a line that says “mlockall” : true, which indicates that memory locking is enabled and working

That’s it. You should be fine to go around – play with your Elasticsearch now.