Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

yq wont recognize single quotation marks around the field when trying to split large YAML file into smaller ones

I have a large YAML file containing info about video games.

---
-
    Name: Metal Gear Solid HD Collection
    Console: PS3
    Genre: Stealth 
    Progression: Unplayed
    Maintenance: Perfect!
-
    Name: Metal Gear Solid 4 Guns of the Patriots
    Console: PS3
    Genre: Stealth 
    Progression: Unplayed
    Maintenance: Perfect!
-
    Name: Metal Gear Rising Revengeance
    Console: PS3
    Genre: Power Fantasy
    Progression: Compleat (Blade Wold DLC to play)
    Maintenance: Perfect!
-

I am trying to split the large file into a file per game. I found a great solution to this with using:

yq e GameCollection.yaml -s .Name

But this gives the following error

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

Error: cannot index array with 'Name' (strconv.ParseInt: parsing "Name": invalid syntax)

I’m assuming that this is because it is trying to treat Name as a string so I wrap it in single quotes to make it more like a String. I found this solution here, the comments in the higheste upvoted answer allowed me to see a different error message and move past this issue.
How to split a YAML into multiple files with a proper name

yq e GameCollection.yaml -s '.Name'

This gives me a different Error and I think its because Name is not a unique identifyer as there are sometimes the same game on multple consoles (such as Skyrim)

Error: bad split document expression: 1:1: invalid input text "'.Name'"

I found, what I thought was a great solution to this, by adding an index so that duplicat names can just incrament! I found that solution here:
yq split kinds and unique name: -s is not keeping unique kind and metadata.name

yq e GameCollection.yaml -s '.Name + "_" + $index'

But it gives me a similar error

Error: bad split document expression: 1:1: invalid input text "'.Name"

I tried again a few times as I think I just dont understand the syntax

yq e GameCollection.yaml -s '.Name' + "_" + '$index'
Error: bad split document expression: 1:1: invalid input text "'.Name'"

yq e GameCollection.yaml -s '.Name' + "_" + $index
Error: bad split document expression: 1:1: invalid input text "'.Name'"

yq e GameCollection.yaml -s .Name + "_" + '$index'
Error: cannot index array with 'Name' (strconv.ParseInt: parsing "Name": invalid syntax)

It’s almost like it doesnt like the single quotes?

If someone could point me in the direction please help me out!

EDIT:
I am using Windows with the CMD tool to do this

>Solution :

The problem has nothing to do with the use of quotation marks. The cause is much simpler: your file contains documents that are single-element arrays. So there is no .Name at the top level, but it’s a key to the first element of each array. Therefore, you need to specify it like this:

yq e GameCollection.yaml -s .[0].Name

Some background info

Compare your file with the one from the other question you looked at.

The one from the other question:

---
name: first
metadata: a

Your file:

---
-
    Name: Metal Gear Solid HD Collection

Notice the extra - line: it’s significant.

Another way to debug the issue is to convert your file into JSON format, which some people find clearer and helps to show structure more explicitly:

yq -o json GameCollection.yaml

Note all the [ ... ] showing that these are arrays.

Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading