Kafka and Kafka connect command cheatsheet

Published

September 8, 2023

These commands I used many times. This page notes down them for quick reference in future.

Kafka Connect

Kafka connect uses Rest API so we can interact with a simple curl command.

Commands for connectors
#sudo apt instal jq -y
curl -X GET http://localhost:8083/connectors| jq
curl -X GET http://localhost:8083/connectors?expand=status | jq
curl -X GET http://localhost:8083/connectors?expand=info | jq
curl -X GET http://localhost:8083/connectors/<connector-name> | jq
curl -X DELETE http://localhost:8083/connectors/<connector-name> | jq
  1. Get list of all connectors
  2. Get status of all connectors
  3. Get info of all connectors
  4. Get a connector name and use it here to get information of it
  5. Delete a respective connector

To know more on jq read here.

Kafka commands

Commands for Kafka
docker exec -it kafka bin/kafka-topics.sh --list --bootstrap-server localhost:9092
docker exec -it kafka bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic debezium.commerce.products --from-beginning
docker exec -it kafka bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test_debezium.commerce.products --from-beginning | grep -i "\"price\":"
  1. List all topics connected to the bootstrap-server
  2. Consume messages with the console consumer
  3. Consume messages with the console consumer but with a grep filter.

Kafka configuration

kafka configuration
  zookeeper:
    image: debezium/zookeeper:2.4
    container_name: zookeeper
    ports:
      - "2181:2181"
    networks:
      - my_network
  kafka:
    container_name: kafka
    image: debezium/kafka:latest
    ports:
      - "9093:9093"
    environment:
      - ZOOKEEPER_CONNECT=zookeeper:2181
      - KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=LISTENER_INT:PLAINTEXT,LISTENER_EXT:PLAINTEXT
      - KAFKA_ADVERTISED_LISTENERS=LISTENER_INT://kafka:9092,LISTENER_EXT://localhost:9093
      - KAFKA_LISTENERS=LISTENER_INT://0.0.0.0:9092,LISTENER_EXT://0.0.0.0:9093
      - KAFKA_INTER_BROKER_LISTENER_NAME=LISTENER_INT
    depends_on:
      - zookeeper
    networks:
      - my_network

networks:
  my_network:

Want to support my blog?

Buy Me A Coffee