Download all messages in a topic to a csv file

[graciasrochelle]

Related open issues: #1183

Download all messages in a topic to a csv file:

Topics._.akhq.io.-.Google.Chrome.2023-03-15.23-19-57.mp4

[graciasrochelle]

Hi @tchiotludo, This pr is ready to be reviewed. Could I please have some feedback on it? Thanks in advance

[tchiotludo]

@graciasrochelle

• can you move the select at the beginning of the rows please?
• can you add the download button on the button bar and changes it to a drop down (with choice of CSV or JSON new line format), disabled by default if no checkbox is selected ?

By this way the functionality will be more visible

[graciasrochelle]

Thanks for the feedback @tchiotludo! I will make the changes. Thanks

[graciasrochelle]

Hi @tchiotludo, Updated PR. Could you have a look when you have sometime? Thanks in advance

[graciasrochelle]

Sorry @tchiotludo I had to fix a code quality issue. Could I get an approval again? Thanks

[graciasrochelle]

Hi @tchiotludo Could you approve again? Sorry had to make a file change. I ran it locally, the build passes now.

[graciasrochelle]

All builds have passed. Is it possible to merge these changes? thanks in advance

[tchiotludo]

hey @graciasrochelle, almost done but you need to move the button on the bottom bar please :

thanks 👍

[graciasrochelle]

Sorry I must have missed that. I will make the change

[graciasrochelle]

Hi @tchiotludo, I have updated my pr. It's ready to be reviewed again. Thanks in advance

[Shekark8]

Hi @tchiotludo thank you for the amazing tool. can you please review this PR.

[rajeshsub]

Hi @tchiotludo I am interested in this feature as well and will help if you could please approve the PR. Thank you so much for AKHQ!

[graciasrochelle]

Hi @tchiotludo Sorry to bother you. But any chance you can approve this PR. We wanted to use this feature once merged.

[rajeshsub]

Thank you so much for this feature @graciasrochelle and thank you @tchiotludo for approving the PR. :-)

[AlexisSouquiere]

@graciasrochelle I think there is an issue with the download feature.
Why did you add this when you build the messages to export array ?

allData.push('\n');

When I test it (CSV or JSON) it leads to wrong file format because there is extra commas for each line except the 1st one due to this \n.

Moreover I think the csv or json option can be confusing because exporting in CSV or JSON produces the same file. It's just the extension that changes

@tchiotludo FYI

[graciasrochelle]

Hi @AlexisSouquiere Thanks for flagging this. Maybe I misunderstood the export to csv requirement. I will update to remove the new line. And also try and edit the csv file to be more meaningful. Will update it this weekend. Sorry for the inconvenience caused.

[graciasrochelle]

@graciasrochelle I think there is an issue with the download feature. Why did you add this when you build the messages to export array ?

allData.push('\n');

When I test it (CSV or JSON) it leads to wrong file format because there is extra commas for each line except the 1st one due to this \n.

Moreover I think the csv or json option can be confusing because exporting in CSV or JSON produces the same file. It's just the extension that changes

@tchiotludo FYI

Hi @AlexisSouquiere, I was having a look at the CSV file, I am not sure if much can be done. I don't know if the headers are static or dynamic. If we know what the headers will be then the file can be made more relevant. What are your thought on this?

[AlexisSouquiere]

For me headers are not important. If people wants raw data (not JSON I mean) to be exported, CSV is a good option we will just create a file with one record by row. Then people can use the file as they like. The file has a CSV extension but it can be CSV or plain text inside.

JSON export can be interesting to export JSON/Avro records but the file content has to be JSON-compliant. So we should build it as a JSON array and each array item is one of the records you want to export

[
   { "field1": 1, "field2", "xxx" }, // 1st record
   { "field1": 1, "field2", "xxx" } // 2nd record
]

[jheinitz]

Hello!

I opened the issue regarding the download of all messages (#1183) some time ago. Thanks @graciasrochelle for implementing this!

I want to share my thoughts on the discussion regarding the messge format. I agree with @AlexisSouquiere that we should either create a simple plain CSV file without any JSON elements OR a JSON-compliant file.

What we have now is better than nothing and I don't know how difficult it is to create a JSON-compliant file, but it would be perfect to have the option to choose between the two options and create two different files.

Thanks again and kind regards

Jens

[AlexisSouquiere]

@graciasrochelle I have created #1442 to fix the issues I mentionned.
Now exporting in JSON will creates a JSON-compliant file (JSON array with parsed messages - warning message will be displayed is messages are not in JSON).
The CSV export can export all the format and we won't have any extra commas for each message

JSON export

Same topic but in CSV format

CSV topic

@tchiotludo it was the last comment that I had regarding the next version release