Add item to list in yq if it doesn't already exist

June 12, 2024

Using mikefarah’s yq v4 I want to conditionally add an IP address to a list within a yml file if it does not exist in the list already.

Given a file like:

servers:
  web:
    - 1.1.1.1
  jobs:
    - 2.2.2.2

If 1.1.1.1 was attempted to be added it should not change, if 3.3.3.3 was added to web then it should become

servers:
  web:
    - 1.1.1.1
    - 3.3.3.3
  jobs:
    - 2.2.2.2

I tried a couple of options, hoping to find something that went along the lines of append if not exists but didn’t see that option.

I eventually settled on an update with append & unique.

yq '.servers.web = (.servers.web + ["3.3.3.3"]  | unique)' test.yml

And for removal I have:

yq '.servers.web = (.servers.web - ["3.3.3.3"])' test.yml

Which seems to work fine.

Any yq experts have some advice on a more clear way of describing the first one, or is that the most clear way of doing it?

>Solution :

That’s basically how you’d do it. Iterating through the items and processing them directly could be a tad more efficient, but unnecessarily complicated to write, and with negligible benefit for such short arrays.

A thing to optimize, though: You could use the update operator |=, so the context is already set, removing some duplicate code. You could even remove the array brackets as yq will automatically add items to arrays correctly (it might, however, be counter-intuitive wrt the - case, where this is not possible):

yq '.servers.web |= (. + ["1.1.1.1"]  | unique)'
# or
yq '.servers.web |= (. + "1.1.1.1"  | unique)'

In the - case, you could even go one step further, and use the -= shortcut, as there is no other function to process (like unique with the + case):