Kafka Notes

Kafka:
-------
Go to /home/cloudera

Download Kafka from :
wget http://www-eu.apache.org/dist/kafka/1.0.0/kafka_2.11-1.0.0.tgz


Run Terminal
tar -xvzf kafka_2.11-1.0.0.tgz

cd kafka_2.11-1.0.0
cd config

now pwd is :
/home/cloudera/kafka_2.11-1.0.0/config

gedit server.properties

search and ensure this is there : zookeeper.connect=localhost:2181

search for 2181 and verify it is there or not. if not add it


Now goto ZooKeeper:
$ cd /usr/lib/zookeeper/bin

start ZoopKeeper
sh zkCli.sh


Now goto Kafka installed folder:
cd /home/cloudera/kafka_2.11-1.0.0/bin/

start kafka
nohup sh kafka-server-start.sh /home/cloudera/kafka_2.11-1.0.0/config/server.properties &

tail -f nohup.outt  -

sh kafka-topics.sh --create --topic Payment_BLR --zookeeper quickstart.cloudera:2181 --replication-factor 1 --partitions 3

sh kafka-console-producer.sh --broker-list quickstart.cloudera:9092 --topic Payment_BLR


sh kafka-console-consumer.sh --zookeeper quickstart.cloudera:2182 --topic mytopic --from-beginning


-----------------------------------------------  May be later use ------------------------------------------------
Here I got : JAVA Unsupported major.minor version 51.0 exception

so, trying to resolve it

Network Issue

checking the mirrorlist server is accessible.

$ ping mirrorlist.centos.org

If you might meet the error message below.

ping: unknown host mirrorlist.centos.org

To solve this, adding a valid nameserver into resolv.conf (use sudo if you are not root user)
$  su
$  nameserver 8.8.8.8 > /etc/resolv.conf

install wget :
sudo yum install wget

install java:
sudo yum install java-1.8.0-openjdk
----------------------------------------- For Future Use -------------------------------------


start kafka
nohup sh kafka-server-start.sh /home/cloudera/kafka_2.11-2.0.0/config/server.properties &

tail -f nohup.out
https://downloads.cloudera.com/demo_vm/vmware/cloudera-quickstart-vm-5.13.0-0-vmware.zip


How to access Windows Shared folder contents within VM?
Devices - Network  - Network Settings - Shared Folders - Add (+) - select windows shared folder location
Install : Insert Guest Additions CD Image
Open Terminal within Cent OS

mount -t vboxsf Tut Tutorials