ElasticSearch(3)Version Upgrade and Cluster

ElasticSearch(3)Version Upgrade and Cluster

Haha, the first version I start with Elastic Search is 1.4.0.

Last time, I install the softwares are version 6.2.4. Right now, it is 7.0.1. Haha. Let’s try the latest on my systems.

First of All, on MAC
https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.0.1-darwin-x86_64.tar.gz

Unzip the file and install in the default directory
> sudo ln -s /Users/hluo/tool/elasticsearch-7.0.1 /opt/elasticsearch-7.0.1
> sudo ln -s /opt/elasticsearch-7.0.1 /opt/elasticsearch

It is already in my path
export PATH=/opt/elasticsearch/bin:$PATH

This command is coming from he old version, it does not be needed here now.
> bin/elasticsearch-plugin install x-pack
ERROR: this distribution of Elasticsearch contains X-Pack by default

Start Elasticsearch with this command
> bin/elasticsearch

After start, visit the Web Page
http://localhost:9200/

Download and Install kibana
https://artifacts.elastic.co/downloads/kibana/kibana-7.0.1-darwin-x86_64.tar.gz

> sudo ln -s /Users/hluo/tool/kibana-7.0.1 /opt/kibana-7.0.1
> sudo ln -s /opt/kibana-7.0.1 /opt/kibana

Kibana is added to my path as well
export PATH=/opt/kibana/bin:$PATH

Edit and Check the configuration
> vi config/kibana.yml
elasticsearch.hosts: ["http://localhost:9200"]

Start the Kibana
> bin/kibana

Visit the WebPage
http://localhost:5601/app/kibana

Elastic Search and Kibana on Ubuntu
Find the Elastic Search download here
https://www.elastic.co/downloads/elasticsearch

> wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.0.1-linux-x86_64.tar.gz

Find the Kibana download from here
https://www.elastic.co/downloads/kibana

> wget https://artifacts.elastic.co/downloads/kibana/kibana-7.0.1-linux-x86_64.tar.gz

Unzip these 2 files and place in the working directory
> sudo ln -s /home/carl/tool/elasticsearch-7.0.1 /opt/elasticsearch-7.0.1
> sudo ln -s /home/carl/tool/kibana-7.0.1 /opt/kibana-7.0.1

> sudo ln -s /opt/elasticsearch-7.0.1 /opt/elasticsearch
> sudo ln -s /opt/kibana-7.0.1 /opt/kibana

When I change the binding IP for Elastic Search, I get error when I start the instance.

> cat config/elasticsearch.yml
network.host: 192.168.56.101
http.port: 9200

bound or publishing to a non-loopback address, enforcing bootstrap checks

Add these to the configuration works
transport.host: localhost
transport.tcp.port: 9300
http.port: 9200
network.host: 0.0.0.0

Kibana Network Configuration
The configuration file is as follow
> cat config/kibana.yml
server.host: "0.0.0.0"
elasticsearch.hosts: ["http://localhost:9200"]

After that, we can visit these pages:
http://ubuntu-master:9200/
http://ubuntu-master:5601

Here is one Compose Configuration to Run elasticsearch1, elasticsearch2, elasticsearch3 and kibana

docker-compose.yml
version: '3.7'
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.0.1
    container_name: elasticsearch1
    environment:
      - node.name=elasticsearch1
      - cluster.name=docker-cluster
      - cluster.initial_master_nodes=elasticsearch1
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512M -Xmx512M"
      - http.cors.enabled=true
      - http.cors.allow-origin=*
      - network.host=_eth0_
    ulimits:
      nproc: 65535
      memlock:
        soft: -1
        hard: -1
    cap_add:
      - ALL
    # privileged: true
    deploy:
      replicas: 1
      update_config:
        parallelism: 1
        delay: 10s
      resources:
        limits:
          cpus: '1'
          memory: 512M
        reservations:
          cpus: '1'
          memory: 512M
      restart_policy:
        condition: on-failure
        delay: 5s
        max_attempts: 3
        window: 10s
    volumes:
      # - "./data/logs:/var/log"
      # - "./data/loc_esdata1:/usr/share/elasticsearch/data"
      - type: volume
        source: logs
        target: /var/log
      - type: volume
        source: loc_esdata1
        target: /usr/share/elasticsearch/data
    networks:
      - elastic
      - ingress
    ports:
      - 9200:9200
      - 9300:9300

  elasticsearch2:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.0.1
    container_name: elasticsearch2
    environment:
      - node.name=elasticsearch2
      - cluster.name=docker-cluster
      - cluster.initial_master_nodes=elasticsearch1
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512M -Xmx512M"
      - "discovery.zen.ping.unicast.hosts=elasticsearch1"
      - http.cors.enabled=true
      - http.cors.allow-origin=*
      - network.host=_eth0_
    ulimits:
      nproc: 65535
      memlock:
        soft: -1
        hard: -1
    cap_add:
      - ALL
    # privileged: true
    deploy:
      replicas: 1
      update_config:
        parallelism: 1
        delay: 10s
      resources:
        limits:
          cpus: '1'
          memory: 512M
        reservations:
          cpus: '1'
          memory: 512M
      restart_policy:
        condition: on-failure
        delay: 5s
        max_attempts: 3
        window: 10s
    volumes:
      # - "./data/logs:/var/log"
      # - "./data/loc_esdata2:/usr/share/elasticsearch/data"     
      - type: volume
        source: logs
        target: /var/log
      - type: volume
        source: loc_esdata2
        target: /usr/share/elasticsearch/data
    networks:
      - elastic
      - ingress
    ports:
      - 9201:9200

  elasticsearch3:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.0.1
    container_name: elasticsearch3
    environment:
      - node.name=elasticsearch3
      - cluster.name=docker-cluster
      - cluster.initial_master_nodes=elasticsearch1
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512M -Xmx512M"
      - "discovery.zen.ping.unicast.hosts=elasticsearch1"
      - http.cors.enabled=true
      - http.cors.allow-origin=*
      - network.host=_eth0_
    ulimits:
      nproc: 65535
      memlock:
        soft: -1
        hard: -1
    cap_add:
      - ALL
    # privileged: true
    deploy:
      replicas: 1
      update_config:
        parallelism: 1
        delay: 10s
      resources:
        limits:
          cpus: '1'
          memory: 512M
        reservations:
          cpus: '1'
          memory: 512M
      restart_policy:
        condition: on-failure
        delay: 5s
        max_attempts: 3
        window: 10s
    volumes:
      # - "./data/logs:/var/log"
      # - "./data/loc_esdata3:/usr/share/elasticsearch/data"     
      - type: volume
        source: logs
        target: /var/log
      - type: volume
        source: loc_esdata3
        target: /usr/share/elasticsearch/data
    networks:
      - elastic
      - ingress
    ports:
      - 9202:9200

  kibana:
    image: docker.elastic.co/kibana/kibana:7.0.1
    container_name: kibana
    environment:
      SERVER_NAME: localhost
      ELASTICSEARCH_URL: http://elasticsearch1:9200/
    ports:
      - 5601:5601
    volumes:
      - type: volume
        source: logs
        target: /var/log
    ulimits:
      nproc: 65535
      memlock:
        soft: -1
        hard: -1
    cap_add:
      - ALL
    deploy:
      replicas: 1
      update_config:
        parallelism: 1
        delay: 10s
      resources:
        limits:
          cpus: '1'
          memory: 512M
        reservations:
          cpus: '1'
          memory: 512M
      restart_policy:
        condition: on-failure
        delay: 30s
        max_attempts: 3
        window: 120s
    networks:
      - elastic
      - ingress
     
  headPlugin:
    image: 'mobz/elasticsearch-head:5'
    container_name: head
    ports:
      - '9100:9100'
    networks:
      - elastic

volumes:
  loc_esdata1:
  loc_esdata2:
  loc_esdata3:
  logs:

networks:
  elastic:
  ingress:

Then we can build and run
> docker-compose up -d

Start the service
> docker-compose start


Some other command
# up
docker-compose up -d

# down
docker-compose down

# down and remove volume
docker-compose down -v

# stop
docker-compose stop

# pause
docker-compose pause

# start
docker-compose start

# remove
docker-compose rm

Then we can visit
http://localhost:5601/app/kibana#/home?_g=()

Cluster Info
http://localhost:9100/

Elastic search node
http://localhost:9201/

Near Realtime(NRT) - Elasticsearch is a near-realtime search platform. There is a slight latency (normally one second) from the time you index a document until the time it becomes searchable.

Cluster - multiple nodes with a cluster name.
Node - its name is an UUID. ( a random Universally Unique IDentifier )  A note can be configured to join a specific Cluster by the cluster name.
            If you start the first node, it will by default form a new single-node cluster named elasticsearch.
Index - An index is a collection of documents. An index is identified by a name( that must be all lowercase)
Document - A document is a basic unit of information that can be indexed.
Shards & Replicas - Elasticsearch provides the ability to subdivide your index into multiple pieces called shards. Replica provides high availability in case a shard/node fails. The number of shards and replicas can be defined per index at the time the index is created. You can change the number of shards for an existing index using the _shrink and _split APIs.

The latest Version I get from the Document, but I think we can only download the 7.0.1 Version.
> curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.2.0-linux-x86_64.tar.gz

I should use this document
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/getting-started-install.html

Start the First Node
> bin/elasticsearch -Ecluster.name=sillycatcluster -Enode.name=elastic1

Here is the information after started
http://ubuntu-master:9200/

{
  "name": "elastic1",
  "cluster_name": "sillycatcluster",
  "cluster_uuid": "szdcUtvTQaK4Taz63-nKGQ",
  "version": {
    "number": "7.0.1",
    "build_flavor": "default",
    "build_type": "tar",
    "build_hash": "e4efcb5",
    "build_date": "2019-04-29T12:56:03.145736Z",
    "build_snapshot": false,
    "lucene_version": "8.0.0",
    "minimum_wire_compatibility_version": "6.7.0",
    "minimum_index_compatibility_version": "6.0.0-beta1"
  },
  "tagline": "You Know, for Search"
}




References:
https://www.elastic.co/downloads/elasticsearch
https://www.elastic.co/downloads/kibana
https://stackoverflow.com/questions/34661210/how-to-bind-kibana-to-multiple-host-names-ips
https://www.elastic.co/guide/en/elasticsearch/reference/7.x/index.html
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/getting-started-install.html