聚合国内IT技术精华文章,分享IT技术精华,帮助IT从业人士成长

ES系列之Elasticsearch-2.3安装笔记

2017-06-14 11:25 浏览: 6566 次 我要评论(0 条) 字号:

一、安装规划
背景:
某个项目需要三台物理机搭建一个ES集群。
机器大概为24核,128G内存,考虑到ES是纯java项目,存在32G内存限制的问题,所以堆内内存一般分配小于等于32G,非堆内内存一般是和堆内内存size一致。
假如我们一个实例堆内内存分配32G,非堆内内存32G,也就是说一个实例会使用64G内存,单机安装两个实例,需要128G内存,这样系统没有可用内存了。
考虑到系统也需要使用一些内存,我们将给每个实例规划25G堆内内存。

实例名称        实例IP                实例端口       软件路径                                                            数据文件路径
instance1    10.16.66.32            9200            /home/bigdata/elasticsearch-2.3.3                    /home/bigdata/es_data
instance2    10.16.66.32            9201            /home/bigdata/elasticsearch-2.3.3_9201         /home/bigdata/es_data_9201
instance3    10.16.66.33            9200            /home/bigdata/elasticsearch-2.3.3                    /home/bigdata/es_data
instance4    10.16.66.33            9201            /home/bigdata/elasticsearch-2.3.3_9201         /home/bigdata/es_data_9201
instance5    10.16.66.34            9200            /home/bigdata/elasticsearch-2.3.3                    /home/bigdata/es_data
instance6    10.16.66.35            9201            /home/bigdata/elasticsearch-2.3.3_9201         /home/bigdata/es_data_9201

二、系统环境设置
1、硬件需求
ES是一个CPU,内存,磁盘使用比较均衡的软件,所以这三个资源不能有明显的瓶颈,不然会影响整体性能。
ES是一个分布式系统,系统自身可以很好的保证数据的不丢失和一致性,所以不用在单台机器上做磁盘RAID,反而会降低磁盘空间利用率和IO性能。

2、内核优化
ES是一个常驻内存的服务,所以需要提前分配及锁定内存,避免内存交换,影响性能。另外就是常见的文件描述符,进程数,tcp连接快速释放和循环利用等参数。
/etc/sysctl.conf 文件如下

kernel.panic = 10
kernel.panic_on_oops = 10
kernel.sysrq = 0
kernel.ctrl-alt-del = 1
kernel.core_pattern = /dev/null
kernel.shmmax = 50696028160
kernel.shmall = 12376960
kernel.msgmnb = 65536
kernel.msgmax = 65536
fs.file-max = 819200
vm.swappiness = 1
vm.dirty_ratio = 60
vm.dirty_background_ratio = 5
vm.min_free_kbytes = 4950780
vm.overcommit_memory = 1
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_dsack = 1
net.ipv4.tcp_sack = 1
net.ipv4.tcp_fack = 1
net.ipv4.conf.all.forwarding = 0
net.ipv4.conf.default.forwarding = 0
net.ipv4.ip_forward = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv4.conf.all.secure_redirects = 0
net.ipv4.conf.default.secure_redirects = 0
net.ipv4.conf.all.bootp_relay = 0
net.ipv4.conf.all.proxy_arp = 0
net.ipv4.icmp_echo_ignore_broadcasts = 1
net.ipv4.icmp_ignore_bogus_error_responses = 1
net.ipv4.tcp_rfc1337 = 1
net.ipv4.tcp_congestion_control = cubic
net.ipv4.tcp_ecn = 2
net.ipv4.tcp_reordering = 3
net.ipv4.tcp_retries2 = 8
net.ipv4.tcp_retries1 = 3
net.ipv4.tcp_no_metrics_save = 1
net.ipv4.tcp_slow_start_after_idle = 0
net.ipv4.tcp_fin_timeout = 10
net.ipv4.tcp_keepalive_time = 300
net.ipv4.tcp_keepalive_probes = 5
net.ipv4.tcp_keepalive_intvl = 15
net.ipv4.ip_local_port_range = 4096 65535
net.core.netdev_max_backlog = 16384
net.ipv4.tcp_max_syn_backlog = 65536
net.core.somaxconn = 32768
nee.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_adv_win_scale = 2
net.ipv4.tcp_rmem = 4096 131072 16777216
net.core.rmem_default = 131072
net.core.rmem_max = 16777216
net.ipv4.tcp_wmem = 4096 131072 16777216
net.core.wmem_default = 131072
net.core.wmem_max = 16777216
net.core.optmem_max = 65536
net.ipv4.tcp_timestamps = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_max_tw_buckets = 20000
net.ipv4.udp_rmem_min = 16384
net.ipv4.udp_wmem_min = 16384
net.ipv4.route.flush = 1
net.ipv4.rt_cache_rebuild_count = -1
net.netfilter.nf_conntrack_max = 10000000
net.netfilter.nf_conntrack_tcp_timeout_established = 360
net.netfilter.nf_conntrack_tcp_timeout_fin_wait = 60
net.netfilter.nf_conntrack_tcp_timeout_syn_recv = 13
net.netfilter.nf_conntrack_tcp_timeout_time_wait = 60
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1

/etc/security/limits.conf文件如下

* soft nofile 1048576
* hard nofile 1048576
* soft nproc 1048576
* hard nproc 1048576
* soft memlock unlimited
* hard memlock unlimited

3、JDK安装
使用JDK8

http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

下载64位二进制版本。jdk-8u131-linux-x64.tar.gz
#tar -zxvf jdk-8u131-linux-x64.tar.gz
#mkdir -p /home/jm1/jdk
#mv jdk1.8.0_131/ /home/jm1/jdk/
# cat /etc/profile.d/jdk.sh
#!/bin/bash
export JAVA_HOME=/home/jm1/jdk/jdk1.8.0_131
export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$CLASSPATH
export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$HOME/bin:$PATH
# chmod 755 /etc/profile.d/jdk.sh
由于我们公司初始化系统对profile做了大量改动,所以需要在/etc/bashrc在导入一次。
在/etc/bashrc最后加入一行
source /etc/profile.d/jdk.sh

然后退出重新登陆,执行
#java -version
java version “1.8.0_131″
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

4、cgorup安装配置
稍后补充

三、下载软件

https://www.elastic.co/downloads/elasticsearch

将软件分发到每台机器的/tmp/目录

四、安装ES
useradd bigdata
mkdir /home/bigdata/es_data/
mkdir /home/bigdata/es_logs/
mkdir /home/bigdata/es_data_9201/
mkdir /home/bigdata/es_logs_9201/

cp -r elasticsearch-2.3.3/ /home/bigdata/elasticsearch-2.3.3
cp -r elasticsearch-2.3.3/ /home/bigdata/elasticsearch-2.3.3_9201/

修改配置文件:
配置文件示例一:
╰─># cat elasticsearch.yml

cluster.name: frist_es

processors: 40

node.name: “frist_es(10.16.66.32)"
node.master: true
node.data: true
node.max_local_storage_nodes: 1

bootstrap.mlockall: true

threadpool.bulk.type: fixed
threadpool.bulk.size: 40
threadpool.bulk.queue_size: 2000

threadpool.index.type: fixed
threadpool.index.size: 40
threadpool.index.queue_size: 2000

threadpool.search.type: fixed
threadpool.search.size: 120
threadpool.search.queue_size: 4000

index.refresh_interval: 10s
index.merge.scheduler.max_thread_count: 1

#path.shared_data:  /home/jm/es/
path.data: /home/bigdata/es_data/
#path.data: /home/jm8/es_data
path.logs: /home/bigdata/es_logs/

network.host: 10.16.66.32
network.publish_host: 10.16.66.32
transport.tcp.port: 9300
http.port: 9200

http.enabled: true
http.cors.allow-origin: "/.*/"
http.cors.enabled: true

index.number_of_shards: 6
index.number_of_replicas: 1

#gateway.type: local
#gateway.recover_after_nodes: 5
#gateway.expected_nodes: 6

cluster.routing.allocation.node_initial_primaries_recoveries: 8
cluster.routing.allocation.node_concurrent_recoveries: 4

indices.recovery.max_bytes_per_sec: 80mb
indices.recovery.concurrent_streams: 5
indices.fielddata.cache.size: 30%
indices.ttl.interval: 43200s
indices.ttl.bulk_size: 100000

discovery.zen.minimum_master_nodes: 2
discovery.zen.ping.timeout: 60s
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts: ["10.16.66.32:9300","10.16.66.32:9301","10.16.66.33:9300","10.16.66.33:9301","10.16.66.34:9300","10.16.66.34:9301"]

index.translog.flush_threshold_size: 512m
index.translog.flush_threshold_ops: 100000
index.translog.sync_interval: 10s

index.search.slowlog.level: INFO
index.search.slowlog.threshold.query.warn: 10s
index.search.slowlog.threshold.query.info: 5s
index.search.slowlog.threshold.query.debug: 2s
index.search.slowlog.threshold.query.trace: 500ms

index.search.slowlog.threshold.fetch.warn: 1s
index.search.slowlog.threshold.fetch.info: 800ms
index.search.slowlog.threshold.fetch.debug: 500ms
index.search.slowlog.threshold.fetch.trace: 200ms

index.indexing.slowlog.level: INFO
index.indexing.slowlog.threshold.index.warn: 10s
index.indexing.slowlog.threshold.index.info: 5s
index.indexing.slowlog.threshold.index.debug: 2s
index.indexing.slowlog.threshold.index.trace: 500ms

action.destructive_requires_name: true

script.groovy.sandbox.enabled: true
script.inline: on
script.indexed: on
script.search: on
script.engine.groovy.inline.aggs: on

配置文件示例二:

cluster.name: frist_es

processors: 40

node.name: “frist_es(10.16.66.32_9201)"
node.master: false
node.data: true
node.max_local_storage_nodes: 1

bootstrap.mlockall: true

threadpool.bulk.type: fixed
threadpool.bulk.size: 40
threadpool.bulk.queue_size: 2000

threadpool.index.type: fixed
threadpool.index.size: 40
threadpool.index.queue_size: 2000

threadpool.search.type: fixed
threadpool.search.size: 120
threadpool.search.queue_size: 4000

index.refresh_interval: 10s
index.merge.scheduler.max_thread_count: 1

#path.shared_data:  /home/jm/es/
path.data: /home/bigdata/es_data_9201/
#path.data: /home/jm8/es_data
path.logs: /home/bigdata/es_logs_9201/

network.host: 10.16.66.32
network.publish_host: 10.16.66.32
transport.tcp.port: 9301
http.port: 9201

http.enabled: true
http.cors.allow-origin: "/.*/"
http.cors.enabled: true

index.number_of_shards: 6
index.number_of_replicas: 1

#gateway.type: local
#gateway.recover_after_nodes: 5
#gateway.expected_nodes: 6

cluster.routing.allocation.node_initial_primaries_recoveries: 8
cluster.routing.allocation.node_concurrent_recoveries: 4

indices.recovery.max_bytes_per_sec: 80mb
indices.recovery.concurrent_streams: 5
indices.fielddata.cache.size: 30%
indices.ttl.interval: 43200s
indices.ttl.bulk_size: 100000

discovery.zen.minimum_master_nodes: 2
discovery.zen.ping.timeout: 60s
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts: ["10.16.66.32:9300","10.16.66.32:9301","10.16.66.33:9300","10.16.66.33:9301","10.16.66.34:9300","10.16.66.34:9301"]

index.translog.flush_threshold_size: 512m
index.translog.flush_threshold_ops: 100000
index.translog.sync_interval: 10s

index.search.slowlog.level: INFO
index.search.slowlog.threshold.query.warn: 10s
index.search.slowlog.threshold.query.info: 5s
index.search.slowlog.threshold.query.debug: 2s
index.search.slowlog.threshold.query.trace: 500ms

index.search.slowlog.threshold.fetch.warn: 1s
index.search.slowlog.threshold.fetch.info: 800ms
index.search.slowlog.threshold.fetch.debug: 500ms
index.search.slowlog.threshold.fetch.trace: 200ms

index.indexing.slowlog.level: INFO
index.indexing.slowlog.threshold.index.warn: 10s
index.indexing.slowlog.threshold.index.info: 5s
index.indexing.slowlog.threshold.index.debug: 2s
index.indexing.slowlog.threshold.index.trace: 500ms

action.destructive_requires_name: true

script.groovy.sandbox.enabled: true
script.inline: on
script.indexed: on
script.search: on
script.engine.groovy.inline.aggs: on

++++++++++++++++++++++++++++++++++
JVM优化
设置JVM内存SIZE
15 if [ "x$ES_MIN_MEM" = "x" ]; then
16 ES_MIN_MEM=25g
17 fi
18 if [ "x$ES_MAX_MEM" = "x" ]; then
19 ES_MAX_MEM=25g
20 fi
21 if [ "x$ES_HEAP_SIZE" != "x" ]; then
22 ES_MIN_MEM=$ES_HEAP_SIZE
23 ES_MAX_MEM=$ES_HEAP_SIZE
24 fi

设置额外的JVM参数
88 JAVA_OPTS=”$JAVA_OPTS -Xss256k -XX:NewRatio=4 -XX:SurvivorRatio=4 -Djna.nosys=true”

五、启动实例
chow -R bigdata:bigdata /home/bigdata
su – bigdata
/home/bigdata/elasticsearch-2.3.3/bin/elasticsearch -d
/home/bigdata/elasticsearch-2.3.3_9201/bin/elasticsearch -d

六、安装插件
su – bigdata
cd elasticsearch-2.3.3
在线安装插件
bin/plugin install hlstudio/bigdesk
bin/plugin install lmenezes/elasticsearch-kopf
bin/plugin install mobz/elasticsearch-head

离线安装插件
./bin/plugin -u file:///tmp/elasticsearch-sql-1.4.6.zip –install sql

注意,在不同的es版本里面,安装插件的命令有所差异。

七、基本使用
在浏览器访问
10.16.66.32:9200/_plugin/kopf //适合运维方面的插件
10.16.66.32:9200/_plugin/head //适合开发管理查询数据的插件
10.16.66.32:9200/_plugin/bigdesk //适合监控集群性能的插件



网友评论已有0条评论, 我也要评论

发表评论

*

* (保密)

Ctrl+Enter 快捷回复