Wrong beginning offset returned resulting in data of length 0 #1130
-
Hello, firstly thank you for this great tool. Problem: I am facing an issue where the data returned from a topic is empty even though the topic has data. Setup: Downloaded standalone jar with configured yml file. I have tried investigating by setting the log to trace and found that it is because during consumption, the offsets are set to the latest for each partition in the topic. However from my understanding of the source code, I thought that instead of subscribing to a topic with a consumer group, it is manually assigning the partitions to a standalone consumer so that the consumer will read all the data from the first offset of each partition. Do correct me if I'm wrong as I'm relatively new to kafka and I was wondering if it is due to some configuration that I should add in the yml file. Thank you! |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 3 replies
-
Hello :) For your response, yes I create a simple consumer group and start consume at the first offset (for OLDEST sort). |
Beta Was this translation helpful? Give feedback.
-
Apologies it's a mistake on my end, I was assuming that an offset > 0 means that there is data in the topic partition. However I realized the messages were only being retained for an hour and thus there was no data. Lesson learnt that offset > 0 does not mean that there is data. |
Beta Was this translation helpful? Give feedback.
Apologies it's a mistake on my end, I was assuming that an offset > 0 means that there is data in the topic partition. However I realized the messages were only being retained for an hour and thus there was no data. Lesson learnt that offset > 0 does not mean that there is data.