Package the Spring Boot jar and create the docker image The two important sections that help in making the magic happen are the Spring Kafka related dependencies and the Avro related dependencies as shown below : spring-kafka $) producer.flush() producer.close(). Let’s open the pom.xml file and look at the maven dependencies that are particularly important in this sample. Download and install Docker and Docker Compose for your OS.Ĭlone this repo to your machine and change directory to spring-kafka-registry. All of our microservices and infrastructure components will be dockerized and run using docker-compose. For this tutorial we will be using the open source components of confluent platform. In our sample application we will build a Spring Boot microservice that produces messages and uses Avro to serialize and push them into Kafka.

View the messages from a Kafka Avro Consumer.

A REST interface to send messages to a Kafka topic with Avro schema.

Creating Avro schema and generating Java classes.

Creating a Kafka Avro Producer using Spring Boot.

The following topics are covered in this tutorial: This is a tutorial for creating a simple Spring Boot application with Kafka and Schema Registry. Code generation as an optional optimization, only worth implementing for statically typed languages. Code generation is not required to read or write data files nor to use or implement RPC protocols.

Simple integration with dynamic languages.

A container file, to store persistent data.

What is Avro?Īccording to : Apache Avro™ is a data serialization system. Schema Registry is a critical component in enforcing data governance in a messaging platform.

This saves a lot of headache for down-stream consumer. Schema Registry enables message producers to comply to a JSON schema and avoid producers from pushing message that are bad in to topics. Simply put, we want to avoid garbage-in-garbage-out scenarios. What is Schema Registry?Īccording to Confluent.io : The Schema Registry stores a versioned history of all schemas and allows for the evolution of schemas according to the configured compatibility settings and expanded Avro support. In this article we will learn how one could use Spring Boot, Apache Kafka and Confluent Inc’s Schema Registry to build such a framework where data governance and quality of messages are ensured. As the usage of enterprise message producers, brokers and consumers grows, it becomes all the more important to make sure the data or payloads produced and consumed are compliant to a known schema.

Spring kafka source code