A json document consists of key value pairs which can be in any order, nested or arranged in arrays. Splunk provides commands for extracting information from structured documents but when dealing with a nested JSON document you’ll have to employ some additional tricks to bring it to a tabular format.

First, have a look at your document and establish how many levels of nesting the JSON document has. In the provided example there are 2 layers. For each of the levels we’ll need to extract some information using spath, aggregate statistics using stats and rename the _raw event to the current level json object. All these steps are repeated for each additional depth of the nested JSON object.

 

| spath
| spath Toplevel{} output=Toplevel
| stats c by Toplevel
| eval _raw=Toplevel

Step 1

| spath
| spath Book{} output=Book
| stats c by Book
| eval _raw=Book

 

Step 2

| makeresults 
| eval _raw="{\"Book\":[{\"Title\":\"ABCDEFGHIJKLMNOPQRSTUVWX\",\"Price\":997.5,\"ISBN\":\"ABCDEFGHIJKLMNOPQRSTUVWX\",\"Genre\":\"ABCD\",\"Author\":[{\"FirstName\":\"ABCDEFGHIJKLM\",\"Surname\":\"ABCDEFGHIJKLMNOPQRS\"},{\"FirstName\":\"ABCD\",\"Surname\":\"ABCDEFGHI\"},{\"FirstName\":\"ABCDEFGHIJKLMNOPQRSTUVWXYZAB\",\"Surname\":\"ABCDEFGHIJKLMNOPQRSTUVWXYZAB\"},{\"FirstName\":\"ABCDEFGHIJ\",\"Surname\":\"ABCDEFGHIJKLMNOPQRSTUVW\"}]},{\"Title\":\"ABCDEFGHIJKLMNO\",\"Price\":115.25,\"ISBN\":\"ABCDEFGHIJKLMNOPQRSTU\",\"Genre\":\"ABCDEFGHIJKLMNOPQRSTUV\",\"Author\":[{\"FirstName\":\"ABCDEFGH\",\"Surname\":\"ABCDEFGH\"}]}]}" 

| spath
| spath Book{} output=Book
| stats c by Book
| eval _raw=Book

| spath 
| spath Author{} output=Author 
| stats c by  Title Price ISBN Genre Author
| eval _raw=Author 

 

Step 3 

| makeresults 
| eval _raw="{\"Book\":[{\"Title\":\"ABCDEFGHIJKLMNOPQRSTUVWX\",\"Price\":997.5,\"ISBN\":\"ABCDEFGHIJKLMNOPQRSTUVWX\",\"Genre\":\"ABCD\",\"Author\":[{\"FirstName\":\"ABCDEFGHIJKLM\",\"Surname\":\"ABCDEFGHIJKLMNOPQRS\"},{\"FirstName\":\"ABCD\",\"Surname\":\"ABCDEFGHI\"},{\"FirstName\":\"ABCDEFGHIJKLMNOPQRSTUVWXYZAB\",\"Surname\":\"ABCDEFGHIJKLMNOPQRSTUVWXYZAB\"},{\"FirstName\":\"ABCDEFGHIJ\",\"Surname\":\"ABCDEFGHIJKLMNOPQRSTUVW\"}]},{\"Title\":\"ABCDEFGHIJKLMNO\",\"Price\":115.25,\"ISBN\":\"ABCDEFGHIJKLMNOPQRSTU\",\"Genre\":\"ABCDEFGHIJKLMNOPQRSTUV\",\"Author\":[{\"FirstName\":\"ABCDEFGH\",\"Surname\":\"ABCDEFGH\"}]}]}" 

| spath
| spath Book{} output=Book
| stats c by Book
| eval _raw=Book

| spath 
| spath Author{} output=Author 
| stats c by  Title Price ISBN Genre Author
| eval _raw=Author 

| spath
| stats c by Title Price ISBN Genre FirstName Surname

 

Reference

The complete JSON

{
  "Book": [
    {
      "Title": "ABCDEFGHIJKLMNOPQRSTUVWX",
      "Price": 997.5,
      "ISBN": "ABCDEFGHIJKLMNOPQRSTUVWX",
      "Genre": "ABCD",
      "Author": [
        {
          "FirstName": "ABCDEFGHIJKLM",
          "Surname": "ABCDEFGHIJKLMNOPQRS"
        },
        {
          "FirstName": "ABCD",
          "Surname": "ABCDEFGHI"
        },
        {
          "FirstName": "ABCDEFGHIJKLMNOPQRSTUVWXYZAB",
          "Surname": "ABCDEFGHIJKLMNOPQRSTUVWXYZAB"
        },
        {
          "FirstName": "ABCDEFGHIJ",
          "Surname": "ABCDEFGHIJKLMNOPQRSTUVW"
        }
      ]
    },
    {
      "Title": "ABCDEFGHIJKLMNO",
      "Price": 115.25,
      "ISBN": "ABCDEFGHIJKLMNOPQRSTU",
      "Genre": "ABCDEFGHIJKLMNOPQRSTUV",
      "Author": [
        {
          "FirstName": "ABCDEFGH",
          "Surname": "ABCDEFGH"
        }
      ]
    }
  ]
}

 

Other methods to convert a complex nested JSON to table:

A method using mvzip mxexpand and split is described on the spath help page of Splunk
https://docs.splunk.com/Documentation/SplunkCloud/latest/SearchReference/Spath

An approach using untable is described in this blog
https://www.crestdatasys.com/blogs/how-to-extract-complex-field-from-nested-json-events-using-splunk-spl/

 

Tags
Submitted by Mitch on