- 浏览: 2550766 次
- 性别:
- 来自: 成都
文章分类
最新评论
-
nation:
你好,在部署Mesos+Spark的运行环境时,出现一个现象, ...
Spark(4)Deal with Mesos -
sillycat:
AMAZON Relatedhttps://www.godad ...
AMAZON API Gateway(2)Client Side SSL with NGINX -
sillycat:
sudo usermod -aG docker ec2-use ...
Docker and VirtualBox(1)Set up Shared Disk for Virtual Box -
sillycat:
Every Half an Hour30 * * * * /u ...
Build Home NAS(3)Data Redundancy -
sillycat:
3 List the Cron Job I Have>c ...
Build Home NAS(3)Data Redundancy
ElasticSearch(4)Manually Elasticsearch Cluster
Follow previous link, start Kibana on ubuntu-master as well.
>bin/elasticsearch -Ecluster.name=laprocluster -Enode.name=elastic1
> bin/kibana
Open Kibana page and go to DEV tool
http://ubuntu-master:5601/app/kibana#/dev_tools/console?_g=()v
In the console, put command
>GET /_cat/health?v
It will give us the cluster information
Node Information Command
>GET /_cat/nodes?v
List the indexes, indices
>GET /_cat/indices?v
Create an index and have a try, create a sample index named customer, just pretty to give the nice JSON response
>PUT /customer?pretty
{
"acknowledged" : true,
"shards_acknowledged" : true,
"index" : "customer"
}
Check the status of the index
>GET /_cat/indices?v
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open customer Y9ieCh28SuSrVLekAMMrng 1 1 0 0 230b 230b
We create an index named customer with 1 primary and 1 replica, there is only 1 node, so it is yellow.
Index and Query a Document
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/getting-started-query-document.html
>PUT /customer/_doc/1?pretty
{
"name": "Carl Luo"
}
PUT /customer/_doc/2?pretty
{
"name": "Leo Luo"
}
PUT /customer/_doc/3?pretty
{
"name": "Angela Yuqi Luo"
}
We can fetch the Records
>GET /customer/_doc/1?pretty
Delete an index
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/getting-started-delete-index.html
>DELETE /customer?pretty
Delete a document
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/getting-started-delete-documents.html
>DELETE /customer/_doc/3?pretty
We can do batch operation as well
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/getting-started-batch-processing.html
Here is some links to load the sample data and do some search around that
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/getting-started-explore-data.html
Manually Set Up Cluster
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/setup.html
Start the ElasticSearch in the daemon
> bin/elasticsearch -d -Ecluster.name=laprocluster -Enode.name=elastic1
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/targz.html
We can save the pid as well
>bin/elasticsearch -d -Ecluster.name=laprocluster -Enode.name=elastic1 -p pid
We can do this later to stop that
>pkill -F pid
Some configuration
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/settings.html
Config files Location
elasticsearch.yml for elasticsearch
jvm.options for JVM
Log4j2.properties for logging
We can change the configuration location if we need
>ES_PATH_CONF=/path/to/my/config ./bin/elasticsearch
Some important configurations
cluster.name: sillycatcluster
node.name: elastic1
path.data: /opt/elasticsearch/data
path.logs: /opt/elasticsearch/logs
transport.host: localhost
transport.tcp.port: 9300
http.port: 9200
network.host: 0.0.0.0
JVM Option is in
> vi config/jvm.options
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/jvm-options.html
Adding Nodes to Cluster
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/add-elasticsearch-nodes.html
Setting Up 2 Other Nodes : ubuntu-dev2 and ubuntu-dev4
> java -version
java version "1.8.0_171"
Download the Oracle JDK from here https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
Copy the file to my virtual machines.
> scp jdk-8u211-linux-x64.tar.gz ubuntu-dev2:/home/carl/install/
Unzip and place in the working directory
> sudo ln -s /home/carl/tool/jdk1.8.0_211 /opt/jdk1.8.0
> sudo ln -s /opt/jdk1.8.0 /opt/jdk
Add this to .profile
export JAVA_HOME="/opt/jdk"
PATH="/opt/jdk/bin:$PATH"
Install Elasticsearch on Node
> wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.0.1-linux-x86_64.tar.gz
> sudo ln -s /home/carl/tool/elasticsearch-7.0.1 /opt/elasticsearch-7.0.1
> sudo ln -s /opt/elasticsearch-7.0.1 /opt/elasticsearch
I am using virtual box with host only adapter, so I can not make it working with 9300
[2019-05-20T14:35:08,876][INFO ][o.e.t.TransportService ] [elastic1] publish_address {192.168.56.101:9300}, bound_addresses {192.168.56.101:9300}
ERROR: [1] bootstrap checks failed
[1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
The vm.max_map_count is the issue, not the IP bind.
http://kael-aiur.com/elk/ES%E9%85%8D%E7%BD%AE%E7%BB%99%E5%A4%96%E9%83%A8%E6%9C%BA%E5%99%A8%E9%80%9A%E8%BF%87ip%E8%AE%BF%E9%97%AE.html
> sudo vi /etc/security/limits.conf
#ftp - chroot /ftp
#@student - maxlogins 4
carl - nofile 65536
> sudo sysctl -w vm.max_map_count=262144
vm.max_map_count = 262144
The configurations are as follow:
On ubuntu-master:
transport.host: ubuntu-master
transport.tcp.port: 9300
http.port: 9200
network.host: 0.0.0.0
discovery.seed_hosts: ["ubuntu-master", "ubuntu-dev2"]
On ubuntu-dev2:
transport.host: ubuntu-dev2
transport.tcp.port: 9300
http.port: 9200
network.host: 0.0.0.0
discovery.seed_hosts: ["ubuntu-master", "ubuntu-dev2"]
We can see the nodes after that
>GET /_cat/nodes?v
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
192.168.56.3 7 86 0 0.19 0.24 0.10 mdi - elastic2
192.168.56.101 33 45 1 0.27 0.25 0.16 mdi * elastic1
Similar to all the configurations, we can add another node as we want.
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
192.168.56.3 13 87 0 0.00 0.05 0.06 mdi - elastic2
192.168.56.103 9 96 30 0.39 0.10 0.03 mdi - elastic3
192.168.56.101 11 45 0 0.08 0.09 0.11 mdi * elastic1
Once the cluster is ready, we can access the data like this
http://ubuntu-dev4:9200/customer/_doc/1?pretty
{"_index": "customer","_type": "_doc","_id": "1","_version": 3,"_seq_no": 4,"_primary_term": 1,"found": true,"_source": {"name": "Karl Luo","gender": "male"}}
References:
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/getting-started-install.html
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/getting-started-create-index.html
http://kael-aiur.com/elk/ES%E9%85%8D%E7%BD%AE%E7%BB%99%E5%A4%96%E9%83%A8%E6%9C%BA%E5%99%A8%E9%80%9A%E8%BF%87ip%E8%AE%BF%E9%97%AE.html
Follow previous link, start Kibana on ubuntu-master as well.
>bin/elasticsearch -Ecluster.name=laprocluster -Enode.name=elastic1
> bin/kibana
Open Kibana page and go to DEV tool
http://ubuntu-master:5601/app/kibana#/dev_tools/console?_g=()v
In the console, put command
>GET /_cat/health?v
It will give us the cluster information
Node Information Command
>GET /_cat/nodes?v
List the indexes, indices
>GET /_cat/indices?v
Create an index and have a try, create a sample index named customer, just pretty to give the nice JSON response
>PUT /customer?pretty
{
"acknowledged" : true,
"shards_acknowledged" : true,
"index" : "customer"
}
Check the status of the index
>GET /_cat/indices?v
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open customer Y9ieCh28SuSrVLekAMMrng 1 1 0 0 230b 230b
We create an index named customer with 1 primary and 1 replica, there is only 1 node, so it is yellow.
Index and Query a Document
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/getting-started-query-document.html
>PUT /customer/_doc/1?pretty
{
"name": "Carl Luo"
}
PUT /customer/_doc/2?pretty
{
"name": "Leo Luo"
}
PUT /customer/_doc/3?pretty
{
"name": "Angela Yuqi Luo"
}
We can fetch the Records
>GET /customer/_doc/1?pretty
Delete an index
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/getting-started-delete-index.html
>DELETE /customer?pretty
Delete a document
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/getting-started-delete-documents.html
>DELETE /customer/_doc/3?pretty
We can do batch operation as well
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/getting-started-batch-processing.html
Here is some links to load the sample data and do some search around that
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/getting-started-explore-data.html
Manually Set Up Cluster
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/setup.html
Start the ElasticSearch in the daemon
> bin/elasticsearch -d -Ecluster.name=laprocluster -Enode.name=elastic1
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/targz.html
We can save the pid as well
>bin/elasticsearch -d -Ecluster.name=laprocluster -Enode.name=elastic1 -p pid
We can do this later to stop that
>pkill -F pid
Some configuration
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/settings.html
Config files Location
elasticsearch.yml for elasticsearch
jvm.options for JVM
Log4j2.properties for logging
We can change the configuration location if we need
>ES_PATH_CONF=/path/to/my/config ./bin/elasticsearch
Some important configurations
cluster.name: sillycatcluster
node.name: elastic1
path.data: /opt/elasticsearch/data
path.logs: /opt/elasticsearch/logs
transport.host: localhost
transport.tcp.port: 9300
http.port: 9200
network.host: 0.0.0.0
JVM Option is in
> vi config/jvm.options
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/jvm-options.html
Adding Nodes to Cluster
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/add-elasticsearch-nodes.html
Setting Up 2 Other Nodes : ubuntu-dev2 and ubuntu-dev4
> java -version
java version "1.8.0_171"
Download the Oracle JDK from here https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
Copy the file to my virtual machines.
> scp jdk-8u211-linux-x64.tar.gz ubuntu-dev2:/home/carl/install/
Unzip and place in the working directory
> sudo ln -s /home/carl/tool/jdk1.8.0_211 /opt/jdk1.8.0
> sudo ln -s /opt/jdk1.8.0 /opt/jdk
Add this to .profile
export JAVA_HOME="/opt/jdk"
PATH="/opt/jdk/bin:$PATH"
Install Elasticsearch on Node
> wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.0.1-linux-x86_64.tar.gz
> sudo ln -s /home/carl/tool/elasticsearch-7.0.1 /opt/elasticsearch-7.0.1
> sudo ln -s /opt/elasticsearch-7.0.1 /opt/elasticsearch
I am using virtual box with host only adapter, so I can not make it working with 9300
[2019-05-20T14:35:08,876][INFO ][o.e.t.TransportService ] [elastic1] publish_address {192.168.56.101:9300}, bound_addresses {192.168.56.101:9300}
ERROR: [1] bootstrap checks failed
[1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
The vm.max_map_count is the issue, not the IP bind.
http://kael-aiur.com/elk/ES%E9%85%8D%E7%BD%AE%E7%BB%99%E5%A4%96%E9%83%A8%E6%9C%BA%E5%99%A8%E9%80%9A%E8%BF%87ip%E8%AE%BF%E9%97%AE.html
> sudo vi /etc/security/limits.conf
#ftp - chroot /ftp
#@student - maxlogins 4
carl - nofile 65536
> sudo sysctl -w vm.max_map_count=262144
vm.max_map_count = 262144
The configurations are as follow:
On ubuntu-master:
transport.host: ubuntu-master
transport.tcp.port: 9300
http.port: 9200
network.host: 0.0.0.0
discovery.seed_hosts: ["ubuntu-master", "ubuntu-dev2"]
On ubuntu-dev2:
transport.host: ubuntu-dev2
transport.tcp.port: 9300
http.port: 9200
network.host: 0.0.0.0
discovery.seed_hosts: ["ubuntu-master", "ubuntu-dev2"]
We can see the nodes after that
>GET /_cat/nodes?v
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
192.168.56.3 7 86 0 0.19 0.24 0.10 mdi - elastic2
192.168.56.101 33 45 1 0.27 0.25 0.16 mdi * elastic1
Similar to all the configurations, we can add another node as we want.
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
192.168.56.3 13 87 0 0.00 0.05 0.06 mdi - elastic2
192.168.56.103 9 96 30 0.39 0.10 0.03 mdi - elastic3
192.168.56.101 11 45 0 0.08 0.09 0.11 mdi * elastic1
Once the cluster is ready, we can access the data like this
http://ubuntu-dev4:9200/customer/_doc/1?pretty
{"_index": "customer","_type": "_doc","_id": "1","_version": 3,"_seq_no": 4,"_primary_term": 1,"found": true,"_source": {"name": "Karl Luo","gender": "male"}}
References:
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/getting-started-install.html
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/getting-started-create-index.html
http://kael-aiur.com/elk/ES%E9%85%8D%E7%BD%AE%E7%BB%99%E5%A4%96%E9%83%A8%E6%9C%BA%E5%99%A8%E9%80%9A%E8%BF%87ip%E8%AE%BF%E9%97%AE.html
发表评论
-
Update Site will come soon
2021-06-02 04:10 1676I am still keep notes my tech n ... -
Stop Update Here
2020-04-28 09:00 315I will stop update here, and mo ... -
NodeJS12 and Zlib
2020-04-01 07:44 474NodeJS12 and Zlib It works as ... -
Docker Swarm 2020(2)Docker Swarm and Portainer
2020-03-31 23:18 367Docker Swarm 2020(2)Docker Swar ... -
Docker Swarm 2020(1)Simply Install and Use Swarm
2020-03-31 07:58 367Docker Swarm 2020(1)Simply Inst ... -
Traefik 2020(1)Introduction and Installation
2020-03-29 13:52 335Traefik 2020(1)Introduction and ... -
Portainer 2020(4)Deploy Nginx and Others
2020-03-20 12:06 429Portainer 2020(4)Deploy Nginx a ... -
Private Registry 2020(1)No auth in registry Nginx AUTH for UI
2020-03-18 00:56 434Private Registry 2020(1)No auth ... -
Docker Compose 2020(1)Installation and Basic
2020-03-15 08:10 373Docker Compose 2020(1)Installat ... -
VPN Server 2020(2)Docker on CentOS in Ubuntu
2020-03-02 08:04 454VPN Server 2020(2)Docker on Cen ... -
Buffer in NodeJS 12 and NodeJS 8
2020-02-25 06:43 384Buffer in NodeJS 12 and NodeJS ... -
NodeJS ENV Similar to JENV and PyENV
2020-02-25 05:14 475NodeJS ENV Similar to JENV and ... -
Prometheus HA 2020(3)AlertManager Cluster
2020-02-24 01:47 421Prometheus HA 2020(3)AlertManag ... -
Serverless with NodeJS and TencentCloud 2020(5)CRON and Settings
2020-02-24 01:46 336Serverless with NodeJS and Tenc ... -
GraphQL 2019(3)Connect to MySQL
2020-02-24 01:48 246GraphQL 2019(3)Connect to MySQL ... -
GraphQL 2019(2)GraphQL and Deploy to Tencent Cloud
2020-02-24 01:48 450GraphQL 2019(2)GraphQL and Depl ... -
GraphQL 2019(1)Apollo Basic
2020-02-19 01:36 325GraphQL 2019(1)Apollo Basic Cl ... -
Serverless with NodeJS and TencentCloud 2020(4)Multiple Handlers and Running wit
2020-02-19 01:19 312Serverless with NodeJS and Tenc ... -
Serverless with NodeJS and TencentCloud 2020(3)Build Tree and Traverse Tree
2020-02-19 01:19 317Serverless with NodeJS and Tenc ... -
Serverless with NodeJS and TencentCloud 2020(2)Trigger SCF in SCF
2020-02-19 01:18 291Serverless with NodeJS and Tenc ...
相关推荐
Elasticsearch(简称ES)是一款强大的开源搜索引擎,广泛应用于数据检索、分析和管理。作为分布式、RESTful风格的搜索和数据分析引擎,Elasticsearch能够提供实时、高可用性以及可扩展的搜索功能。在进行日常的数据...
es-head是一个针对Elasticsearch的可视化操作插件。它提供了一个便捷的操作工具,可以连接Elasticsearch...4.连接 Elasticsearch。在 es-head 插件页面输入连接地址 ip 和端口号后点链接即可。连接后显示绿色代表健康。
(狂神)ElasticSearch快速入门笔记,ElasticSearch基本操作以及爬虫(Java-ES仿京东实战),包含了小狂神讲的东西,特别适合新手学习,笔记保存下来可以多看看。好记性不如烂笔头哦~,ElasticSearch,简称es,es是一个...
多弹性搜索头,对著名的 Elasticsearch Head 的改进 1.保存和存储几个Elasticsearch端点 2.索引选项卡中的更多列 3. 任何请求现在都可以像 /_cat/indices 一样处理 JSON 返回 4. 更简约的外观(更小的字体等...) ...
Cluster是由多个Node组成的,每个Cluster都有一个唯一的集群名称,默认为"elasticsearch"。Index是Elasticsearch中管理数据的顶层单位,相当于数据库中的表,数据以JSON格式存储在Document中。Document是Index中的单...
Elasticsearch(ES)是一种流行的开源全文搜索引擎,它基于Lucene构建,提供了分布式、RESTful风格的搜索和分析引擎服务。在中文环境下,为了实现精确的分词和搜索,我们需要安装适合版本的分词插件,如“elastic...
Elasticsearch 是位于 Elastic Stack 核心的分布式搜索和分析引擎。Logstash 和 Beats 有助于收集、聚合和丰富您的数据并将其存储在 Elasticsearch 中。Kibana 使您能够以交互方式探索、可视化和分享对数据的见解,...
3. 停止Elasticsearch:`/usr/local/elasticsearch/elasticsearch-7.13.4/bin/elasticsearch-stop` 安全性和权限: - Elasticsearch 7.x版本开始,默认启用了安全功能,需要设置用户和角色。可以通过Kibana的...
4. **配置文件**:Elasticsearch的主要配置文件是`elasticsearch.yml`,位于解压后的`config`目录下。你可以根据需求修改集群名称、节点角色、索引分片数量等参数。 5. **启动Elasticsearch**:在解压后的目录下,...
docker run --name elasticsearch7.16.3 -p 127.0.0.1:9200:9200 -p 127.0.0.1:9300:9300 -e "discovery.type=single-node" -v /Users/xingyue/Home/xingyue/学习/工程化/es/elasticsearch.yml:/usr/share/elastic...
根据提供的文件信息,我们可以推断出本篇文章将围绕Elasticsearch 6.2.2版本进行详细介绍,包括其下载方式、主要功能特性以及在实际应用中的常见用途。 ### Elasticsearch简介 Elasticsearch是一款基于Lucene的...
Elasticsearch(ES)是一种流行的开源全文搜索引擎,它基于Lucene库构建,被广泛用于大数据分析、日志聚合、实时搜索和索引等场景。官方提供的测试数据集是检验Elasticsearch功能、性能和稳定性的关键资源,可以帮助...
4. **实体类注解**:为需要存储到Elasticsearch的实体类添加`@Document`注解,并指定索引名称。 ```java @Document(indexName = "users") public class User { @Id private Long id; private String username; ...
- **单节点配置**:在`elasticsearch.yml`中,可以设置`cluster.name`以定义集群名称,保持所有节点的集群名称一致,它们就会自动加入同一集群。 - **内存设置**:根据你的系统资源调整`jvm.options`中的堆大小。...
适用于7.17.1系列,例如Elasticsearch的7.17.12版本。 elasticsearch-analysis-ik 是一个常用的中文分词器,在 Elasticsearch 中广泛应用于中文文本的分析和搜索。下面是 elasticsearch-analysis-ik 分词器的几个...
`start` 用于启动Elasticsearch,这里使用 `su` 命令切换到指定的Elasticsearch用户(例如 `es-admin`),然后进入Elasticsearch的安装目录并执行 `bin/elasticsearch` 文件以后台模式启动服务。`stop` 通过查找并杀...
4. **聚合分析**:学习如何使用 Elasticsearch 的聚合功能进行数据统计和分析,如术语聚合、范围聚合、统计聚合等。 5. **映射与分析**:了解字段映射的重要性,以及如何设置分析器进行文本分词,以满足不同的搜索...
这个名为"基于.netcore搜索封装ElasticSearch.zip"的压缩包,显然包含了一个针对.NET Core平台的Elasticsearch客户端库,方便开发者在.NET Core应用中集成和操作Elasticsearch。 Elasticsearch是一个开源的分布式...
Elasticsearch是一个开源的全文搜索引擎,它以分布式、RESTful服务的方式提供快速、高可用、可扩展的数据搜索和分析能力。这个"elasticsearch-7.4.0-win64.rar"压缩包包含了Elasticsearch 7.4.0版本的Windows 64位...
4. **使用兼容模式**:某些情况下,Elasticsearch可能会提供一种向后兼容的模式,允许旧版本的客户端连接新版本的服务器。检查Elasticsearch的配置文档以获取相关信息。 5. **查阅官方文档**:Spring Data Elastic...