jq: Using stream and not

July 30, 2023

This is likely a shamefully simple problem, but I cannot figure it out. I am trying to use this question to extract some data using the –stream option of jq. Here is my sample json:

{
  "date": "2023-07-30",
  "results":[
    {
      "data": [    
        {"row": [{"key1": "row1", "key2": "row1"}]},
        {"row": [{"key1": "row2", "key2": "row2"}]}
      ]
    },
    {
      "data": [    
        {"row": [{"key1": "row3", "key2": "row3"}]},
        {"row": [{"key1": "row4", "key2": "row4"}]}
      ]
    }
  ]
}

Without stream, I can use the following to extract what I want:

jq -rc ".results[]" my_json.json

and this would give me the desired result:

{"data":[{"row":[{"key1":"row1","key2":"row1"}]},{"row":[{"key1":"row2","key2":"row2"}]}]}
{"data":[{"row":[{"key1":"row3","key2":"row3"}]},{"row":[{"key1":"row4","key2":"row4"}]}]}

This output is one json object per line, just what I would like.

However, if I want to do the same thing using the stream option, I am unable to get the same output.

jq -rc --stream 'fromstream(1|truncate_stream(inputs | select(.[0][0] == "results")))' my_json.json

which gives:

[
  {"data":[{"row":[{"key1":"row1","key2":"row1"}]},{"row":[{"key1":"row2","key2":"row2"}]}]},
  {"data":[{"row":[{"key1":"row3","key2":"row3"}]},{"row":[{"key1":"row4","key2":"row4"}]}]}
]

I feel like I am missing one simple step to getting the answer I need.

>Solution :

You need to truncate_stream by two levels: .results and .[]. Use inputs (and the -n flag) for the input, and fromstream to construct the output:

jq --stream -cn 'fromstream(2|truncate_stream(inputs))'

If you want to specify .results explicitly (necessary if the input contains more than just this one field), use select on the input:

jq --stream -cn 'fromstream(2|truncate_stream(inputs | select(.[0][0] == "results")))'

Output:

{"data":[{"row":[{"key1":"row1","key2":"row1"}]},{"row":[{"key1":"row2","key2":"row2"}]}]}
{"data":[{"row":[{"key1":"row3","key2":"row3"}]},{"row":[{"key1":"row4","key2":"row4"}]}]}