大数据--Logstash介绍

1、Logstash介绍

Logstash是一款轻量级的日志搜集处理框架，可以方便的把分散的、多样化的日志搜集起来，并进行自定义的处理，然后传输到指定的位置，比如某个服务器或者文件。

当然它可以单独出现，作为日志收集软件，你可以收集日志到多种存储系统或临时中转系统，如MySQL，redis，kakfa，HDFS, lucene，solr等并不一定是ElasticSearch。

Logstash使用管道方式进行日志的搜集处理和输出。在logstash中，包括了三个阶段:

输入input --> 处理filter（不是必须的） --> 输出output

 

 

 

 

 

 

What's New?

The Logstash 2.4 release gets new plugin features that are otherwise exclusive of the remaining 2.x series. Some highlights include significant performance gains when ingesting data from Beats, and support for Kafka 0.10. For developers, there's a shiny plugin generator to help you build your next plugin! Learn more.

 

Centralize Data Processing of All Types

Logstash is a data pipeline that helps you process logs and other event data from a variety of systems. With 200 plugins and counting, Logstash can connect to a variety of sources and stream data at scale to a central analytics system.

 

Normalize Varying Schema

Business-critical data is often scattered among different systems, each in its own format. Logstash allows you to parse this data and converge on a common format before inserting it into your analytics datastore of choice.

 

Extend to Custom Log Formats

Most logs written by infrastructure and applications have custom formats. Logstash provides a fast and convenient way to custom logic for parsing these logs at scale.

 

Add Plugins for Custom Sources

Built with extensibility in mind, Logstash provides an API for rapid plugin development by the community. With recent improvements to the plugin ecosystem, contributors can publish new plugins at any time.