flume 1.6安装

1.解压

2.复制

　　cp conf/flume-conf.properties.template conf/flume.conf

　　cp conf/flume-env.sh.template conf/flume-env.sh

3. 修改 JAVA_HOME

　　flume-env.sh

4.检验安装

　　flume-ng version

5.启动flume agent agent1

　　flume-ng agent --conf conf -f /bigdata/flume-1.6/conf/agent1.conf -n agent1 -Dlume.root.logger=DEBUG,console

6.创建指定文件

　　echo "hello world" > /home/hadoop/flume-1.5.0-bin/log.00

7.使用avro-client发送文件

　　flume-ng avro-client -c . -H localhost -p 41414 -F /bigdata/flume-1.6/log.00

8.在 agent1控制台会查看到相关信息

更多详细配置

　

二. 收集 udp

　　udp.conf

　　

# Name the components on this agenta1.sources = r1a1.sinks = k1a1.channels = c1a1.sources.r1.type = syslogudpa1.sources.r1.port = 8283a1.sources.r1.host = cdh101a1.sources.r1.channels = c1 #HDFS PATH#a1.sinks.k1.hdfs.path = hdfs://nameservice:8020/user/root/udp/%Y%m%d/hour=%H#HDFS thread numbersa1.sinks.k1.hdfs.threadsPoolSize=100 # HDFS the sink#a1.sinks.k1.type = hdfsa1.sinks.k1.type = loggera1.sinks.k1.hdfs.useLocalTimeStamp = truea1.sinks.k1.hdfs.filePrefix = udpa1.sinks.k1.hdfs.rollInterval = 0a1.sinks.k1.hdfs.rollSize = 118435456a1.sinks.k1.hdfs.rollCount = 0a1.sinks.k1.hdfs.idleTimeout = 1000a1.sinks.k1.hdfs.fileType = DataStream a1.sinks.k1.hdfs.rollTimerPoolSize = 1#memory sizea1.channels.c1.type = memorya1.channels.c1.transactionCapacity = 100000a1.channels.c1.capacity = 268435456# Bind the source and sink to the channela1.sources.r1.channels = c1a1.sinks.k1.channel = c1