オープンソースログ管理ツールLogstashログシステム環境を構築する手順
1.システム要件：
次のモジュールが必要
Logstash
Elasticsearch
Redis
Nginx
Kibana
サーバー側：
fqdn: dev.kanbier.lan (should be resolvable!)
ip: 10.37.129.8
ダウンロードURL:
http://logstash.net/
2.必要なモジュールをインストール
2.1 elasticsearch redis nginx logstashのインストール
$ vi /etc/yum.repos.d/logstash.repo
[logstash-1.4]
name=logstash repository for 1.4.x packages
baseurl=http://packages.elasticsearch.org/logstash/1.4/centos
gpgcheck=1
gpgkey=http://packages.elasticsearch.org/GPG-KEY-elasticsearch
enabled=1
$ vi /etc/yum.repos.d/elasticsearch.repo
[elasticsearch-1.0]
name=Elasticsearch repository for 1.0.x packages
baseurl=http://packages.elasticsearch.org/elasticsearch/1.0/centos
gpgcheck=1
gpgkey=http://packages.elasticsearch.org/GPG-KEY-elasticsearch
enabled=1
$ vi /etc/yum.repos.d/nginx.repo
[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/centos/$releasever/$basearch/
gpgcheck=0
enabled=1
$ rpm -Uvh http://mirror.1000mbps.com/fedora-epel/6/i386/epel-release-6-8.noarch.rpm
$ yum -y install elasticsearch redis nginx logstash
2.2 Kibanaのダウンロード、設定
$ wget https://download.elasticsearch.org/kibana/kibana/kibana-3.0.0.tar.gz
$ tar -xvzf kibana-3.0.0.tar.gz
$ mv kibana-3.0.0 /usr/share/kibana3
$ vi /usr/share/kibana3/config.js
elasticsearch: “http://dev.kanbier.lan:9200",
default_route : '/dashboard/file/logstash.json’,
webページにアクセス：
$ wget https://raw.github.com/elasticsearch/kibana/master/sample/nginx.conf
$ mv nginx.conf /etc/nginx/conf.d/
$ vi /etc/nginx/conf.d/nginx.conf
server_name dev.kanbier.lan;
2.3 redisを設定
$ vi /etc/redis.conf
bind 10.37.129.8
2.4 Logstashを設定
操作手順：
/var/logディレクトリにファイルを読み書き
5544ポートを開放
logstash-complex.confを編集
$ vi /etc/logstash/conf.d/logstash-complex.conf
input {
file {
type => “syslog"
# Wildcards work, here <img src="http://www.denniskanbier.nl/blog/wp-includes/images/smilies/icon_smile.gif" alt=":)" class="wp-smiley">
path => [ “/var/log/*.log", “/var/log/messages", “/var/log/syslog" ]
sincedb_path => “/opt/logstash/sincedb-access"
}
redis {
host => “10.37.129.8"
type => “redis-input"
data_type => “list"
key => “logstash"
}
syslog {
type => “syslog"
port => “5544"
}
}
filter {
grok {
type => “syslog"
match => [ “message", “%{SYSLOGBASE2}" ]
add_tag => [ “syslog", “grokked" ]
}
}
output {
elasticsearch { host => “dev.kanbier.lan" }
}
サービスが起動
$ service redis start; chkconfig redis on
$ service elasticsearch start; chkconfig –add elasticsearch; chkconfig elasticsearch on
$ service logstash start; chkconfig logstash on
$ service nginx start; chkconfig nginx on
「/etc/rsyslog.conf」を編集
# ### begin forwarding rule ###
# The statement between the begin … end define a SINGLE forwarding
# rule. They belong together, do NOT split them. If you create multiple
# forwarding rules, duplicate the whole block!
# Remote Logging (we use TCP for reliable delivery)
#
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
$WorkDirectory /var/lib/rsyslog
# where to place spool files
$ActionQueueFileName fwdRule1
# unique name prefix for spool files
$ActionQueueMaxDiskSpace 1g
# 1gb space limit (use as much as possible)
$ActionQueueSaveOnShutdown on
# save messages to disk on shutdown
$ActionQueueType LinkedList
# run asynchronously
$ActionResumeRetryCount -1
# infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
*.* @@10.37.129.8:5544
3.ファイアウォールのポートを開放
port 80 (for the web interface)
port 5544 (to receive remote syslog messages)
port 6379 (for the redis broker)
port 9200 (so the web interface can access elasticsearch)
参考URL:
http://www.denniskanbier.nl/blog/logging/installing-logstash-on-rhel-and-centos-6/