Ricardo Avila

jq - a tool for manipulating JSON

 

JSON (JavaScript Object Notation) is a common structured data format. It is often used as a replacement for XML, to transmit data between a server and a browser. JSON transmits human-readable data as objects composed of attribute-value pairs.

Example JSON (https://www.w3schools.com/Js/json_demo.txt):

{
    "name":"John",
    "age":31,
    "pets":[
        { "animal":"dog", "name":"Fido" },
        { "animal":"cat", "name":"Felix" },
        { "animal":"hamster", "name":"Lightning" }
    ]
}

Here, we take a look at the jq UNIX tool for manipulating JSON data.

To get started, we can pipe the json text above into jq using curl:

curl -s https://www.w3schools.com/Js/json_demo.txt | jq

Selecting Attributes

Selecting a single scalar value:

 curl -s https://www.w3schools.com/Js/json_demo.txt | jq '.name'

"John"

Selecting multiple values:

 curl -s https://www.w3schools.com/Js/json_demo.txt | \
 jq '.name, .age'

"John"
31

Selecting a single value from an array (index 0 returns the first element)

curl -s https://www.w3schools.com/Js/json_demo.txt | jq '.pets[0]'

{
  "animal": "dog",
  "name": "Fido"
}
More complicated queries can be achieved by combining dot operators, or using a pipe “ ”. 
# Multiple dot operations
curl -s https://www.w3schools.com/Js/json_demo.txt | \
jq '.pets[0].name'

# Same query using a pipe
curl -s https://www.w3schools.com/Js/json_demo.txt | \
jq '.pets[0] | .name'


"Fido"

Selecting elements from an array when an attribute matches a condition:

curl -s https://www.w3schools.com/Js/json_demo.txt | \
jq '.pets | map(select(.animal == "hamster"))'

[
  {
    "animal": "hamster",
    "name": "Lightning"
  }
]

Select also supports boolean operators:

curl -s https://www.w3schools.com/Js/json_demo.txt | \
jq '.pets | map(select(.animal == "hamster" or .animal == "cat"))'

[
  {
    "animal": "cat",
    "name": "Felix"
  },
  {
    "animal": "hamster",
    "name": "Lightning"
  }
]

Formatting Output

Selected attributes can be formatted:

curl -s https://www.w3schools.com/Js/json_demo.txt | \
jq '.name + " is " + (.age | tostring)'

"John is 31"

Editing JSON

Replace a value:

curl -s https://www.w3schools.com/Js/json_demo.txt | \
jq ' .name = "Jessica" '

i{
  "name": "Jessica",
  "age": 31,
  "pets": [
    {
      "animal": "dog",
      "name": "Fido"
    },
    {
      "animal": "cat",
      "name": "Felix"
    },
    {
      "animal": "hamster",
      "name": "Lightning"
    }
  ]
}

Adding a new key-value pair:

curl -s https://www.w3schools.com/Js/json_demo.txt | \
jq ' .pets += [{ "animal" : "piggy", "name": "Chrorizo" }] '

{
  "name": "John",
  "age": 31,
  "pets": [
    {
      "animal": "dog",
      "name": "Fido"
    },
    {
      "animal": "cat",
      "name": "Felix"
    },
    {
      "animal": "hamster",
      "name": "Lightning"
    },
    {
      "animal": "piggy",
      "name": "Chrorizo"
    }
  ]
}

More extensive modifications:

curl -s https://www.w3schools.com/Js/json_demo.txt | \
jq ' .pets += [{ "animal" : "piggy", "name": "Chrorizo" } ] | 
     .pets[0] += {"breed" : "Chihuahua" } |
     .name = "Ana" |
     .age = 28 |
     .hobbies = "ukulele" '

{
  "name": "Ana",
  "age": 28,
  "pets": [
    {
      "animal": "dog",
      "name": "Fido",
      "breed": "Chihuahua"
    },
    {
      "animal": "cat",
      "name": "Felix"
    },
    {
      "animal": "hamster",
      "name": "Lightning"
    },
    {
      "animal": "piggy",
      "name": "Chrorizo"
    }
  ],
  "hobbies": "ukulele"
}
Search