This article is a list of testing ideas for Kafka pub/sub applications based on my experience of working with such applications in multiple contexts.
To understand more about how Kafka works, do check out this awesome series Gently down the stream by Round robin
While publishing a message
- What if the producer instance can not be created/is terminated?
- What if the producer fails to publish a message
- How does the producer ensure the contract that the consumer is expecting?
- Does the producer need to guarantee the ordering of messages?
While consuming a message
- What if the consumer instance can not be created/is terminated?
- What if the consumer picks up a message that it can't process? How should the consumer process run time exceptions?
- Consumers process messages after a certain delay. if the queue is stuck, the Consumer may end up processing very old messages. does it matter if the consumer processes a message that is very old?
- What if the producer has been publishing messages while the consumer was not processing any messages. Starting a consumer when a topic contains a lot of messages, leads to situation where consumers try to read a large number of messages at once, leading to memory issues. How should consumers process a huge backlog of messages?
- What if the consumer processes the same message twice? Is it important that consumers maintain idempotence?
- How many consumer instances are available within a consumer group? Do we have more consumer instances than the topic partitions?
Cross functional testing ideas
- Observability
- Performance
- Security
This is not a comprehensive list, but a work in progress. I will continue to update this as and when I uncover new scenarios.