Wrong beginning offset returned resulting in data of length 0

[royleochan]

Hello, firstly thank you for this great tool.

Problem: I am facing an issue where the data returned from a topic is empty even though the topic has data.

Setup: Downloaded standalone jar with configured yml file.

I have tried investigating by setting the log to trace and found that it is because during consumption, the offsets are set to the latest for each partition in the topic. However from my understanding of the source code, I thought that instead of subscribing to a topic with a consumer group, it is manually assigning the partitions to a standalone consumer so that the consumer will read all the data from the first offset of each partition.

Do correct me if I'm wrong as I'm relatively new to kafka and I was wondering if it is due to some configuration that I should add in the yml file. Thank you!

[tchiotludo]

Hello :)

• How are you sure that the topic as data?
• Is the data is committed ?
• Maybe you can raise this ?

For your response, yes I create a simple consumer group and start consume at the first offset (for OLDEST sort).

[royleochan]

• How are you sure that the topic has data?
  I have tested with a producer and consumer. I have been producing messages with it and the consumer has been successfully consuming from the topic. Further on inspecting the logs, the offset of the partitions in the topic are > 0. Another observation is that if I run the akhq.jar first and then produce a message to the topic, it is able to pick up the new message with a count of 1. However I want to be able to view all the messages in the topic starting from the first offset.
• Is the data is committed ?
  Yes I believe it is.
• Maybe you can raise this ?
  I have tried to but it doesn't work. I don't think it is a timeout issue as the request is successful but it just returns empty, because the akhq consumer is consuming from the latest offset instead of the earliest.

[royleochan]

Apologies it's a mistake on my end, I was assuming that an offset > 0 means that there is data in the topic partition. However I realized the messages were only being retained for an hour and thus there was no data. Lesson learnt that offset > 0 does not mean that there is data.

[Ixi1974]

I think I was having a similar problem, but using standalone Docker release connected to confluent cloud. I.e. messages not showing up in a topic, even though no filters applied and the summary count (double~~) is greater than zero.

I increased akhq.topic-data.poll-timeout to 10000ms and these started appearing. I'm not sure how or why.

[tchiotludo]

akhq.topic-data.poll-timeout is here for that. I've also seen that confluent cloud can be a little bit longer to fetch data, don't know why