Converting Azure EventHub Capture AVRO to JSON Lines (JSONL)
quickly convert lots of avro files to json using eventhub capture files
One of the notable features of EventHub is EventHub Capture. This automatically captures a set of messages (zero or more) into an Apache Avro File in a Storage Account Container of your choosing.
But using the AVRO files, even human reading, needs a little bit of help. This Python script helps convert each AVRO file that is > 508 bytes to a JSON Lines file alongside the AVRO file. Here's an alternative way to download direct from Storage as well - https://askpythonquestions.com/2021/04/07/how-to-convert-from-an-avro-file-to-a-json-file-which-was-originally-sent-as-raw-json-via-postman-through-azure-event-hub/
What follows is the code, simple README.md along with the python requirements.txt.
