If you are working with Kafka and Zookeeper to create some test cases or for the work in bigger projects, you can install them directly from the respective project website or make an installation with a Docker container. The following hints can help you with your first steps to make progress when working with the tools.
Kafka and Zookeeper within Docker
The examples assumes that the client can access the instance within the Docker at port 33222.
Output the logs
docker logs -f my-project-kafka-c docker logs -f my-project-zk4kafka-c
Output all messages for a specific topic
docker exec -it my-report-kafka-c /usr/bin/kafka-console-consumer --bootstrap-server localhost:33222 --topic TOPICNAME --from-beginning
Kafka and Zookeeper as direct installation
Download Apache Kafka from the respective Apache project website. Extract the package and you can execute the following tasks.
The following examples expect that you are within the respective folder of Apache Kafka. The version of Apache Kafka is kafka_2.12-1.0.0.
Start the Apache Zookeeper and Apache Kafka locally
bin/zookeeper-server-start.sh config/zookeeper.properties
bin/kafka-server-start.sh config/server.properties
Create a topic
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic TOPICNAME
Sent a message to the topic
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic TOPICNAME
Show all received messages for a specific topic
bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic TOPICNAME --from-beginning
Show the existing topics
bin/kafka-topics.sh --list --zookeeper localhost:2181
Use local Apache Kafka instance to access data within Apache Kafka instance of Docker
The examples assumes that the client can access the Apache Kafka instance within the Docker at port 33222 and the Apache Zookeeper instance within Docker at port 33272.
Consume messages for a specific topic
bin/kafka-console-consumer.sh --bootstrap-server localhost:33222 --topic TOPICNAME --from-beginning
Produce messages to the instance within Docker
bin/kafka-console-producer.sh --broker-list localhost:33222 --topic TOPICNAME
List the topics from the Apache Zookeeper instance within Docker
bin/kafka-topics.sh --list --zookeeper localhost:33272
Additional tools
Look for processes that occur on the port for the local Apache Kafka within Docker
The example assumes that the client can access the Apache Kafka instance within the Docker at port 33222.
netstat -tulpe | grep 33222
Remove everything that was created within docker
docker-compose down -v