Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

Is it possible to explode JSON array on ingestion stage?

Azure Data Explorer is receiving data through Event Hub subscription. The payload is compressed JSON of the type:

{
  "foo": "bar",
  "why": 42,
  "data": [
    {"field1": "abc", "field2": 123},
    {"field1": "xyz", "field2": 456},
    {"field1": "pqr", "field2": 789}
  ]
}

I need to convert data into tabular format:

filed1 field2
-------------
abc    123
xyz    456
pqr    789

or even better:

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

foo    why    field1 field2
---------------------------
bar    42     abc    123
bar    42     xyz    456
bar    42     pqr    789

I need to create an ingestion mapping, which is a case of data mapping. Looking at the path syntax, I cannot figure out how to create such a mapping.

Is it possible? If not, what is the best way to set up such a transformation during ingestion?

>Solution :

You can achieve that using an update policy.

There’s an example you can follow here: https://docs.microsoft.com/en-us/azure/data-explorer/ingest-json-formats?tabs=kusto-query-language#ingest-json-records-containing-arrays

Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading