Lua : How to compare two json/tables and report on the differences?

I hope someone can help me with this –

I make a number of API calls to various services, all of which return a structured JSON response. Normally I just convert to a table, and extract the one or two values i need, but I’d really like to be able to do a full comparison check to see everything that’s changed since I retrieved the last one.

I’ve already gone down the path of writing a block of code to specifically compare each entry within the associated tables generated, (I’m on Lua version 5.1) but I wanted to ask is there a more dynamic way of doing this, one which is not so rigid that I could even apply against any other json calls (2 x Lua tables) i want to compare?

(I’ve looked at various SO posts about comparing Lua tables but most just look to report if there’s a difference overall (true/false), none of them seem to itemise the differences, or can adapt to different json/table structures – which is important considering the variety of json structures API services return)

Below is an example api json/response, and a sample of the code I’ve written to compare an entry in both tables and report the differences…

Is there a dynamic version that can do the same thing, avoiding the need to script in detail, every comparison step ?

local json = require("dkjson")
local snapshot1 = [[
{
   "ActTime" : 1501360852,
   "ServerTime" : "2017-07-29 22:40:52",
   "Sunrise" : "05:50",
   "Sunset" : "21:28",
   "result" : [
      {
         "AddjMulti" : 1.0,
         "AddjValue" : 0.0,
         "BatteryLevel" : 255,
         "Data" : "73 Lux",
         "Description" : "",
         "HardwareID" : 4,
         "HardwareName" : "Dummies",
         "HardwareType" : "Dummy",
         "ID" : "82089",
         "LastUpdate" : "2017-07-29 21:16:22",
         "Name" : "ESP8266C_Licht1",
         "Notifications" : "false",
         "ShowNotifications" : true,
         "SignalLevel" : "334",
         "Timers" : "false",
         "Type" : "Lux",
         "YOffset" : "0",
         "idx" : "89"
      }
   ],
   "status" : "OK",
   "title" : "Devices"
}
]]

local snapshot2 = [[
{
   "ActTime" : 1501360852,
   "ServerTime" : "2017-07-31 11:40:52",
   "Sunrise" : "05:52",
   "Sunset" : "21:29",
   "result" : [
      {
         "AddjMulti" : 1.0,
         "AddjValue" : 0.0,
         "BatteryLevel" : 255,
         "Data" : "73 Lux",
         "Description" : "",
         "HardwareID" : 4,
         "HardwareName" : "Dummies",
         "HardwareType" : "Dummy",
         "ID" : "82089",
         "LastUpdate" : "2017-07-30 21:16:22",
         "Name" : "ESP8266C_Licht1",
         "Notifications" : "false",
         "ShowNotifications" : true,
         "SignalLevel" : "545",
         "Timers" : "true",
         "Type" : "Lux",
         "YOffset" : "0",
         "idx" : "89"
      }
   ],
   "status" : "OK",
   "title" : "Devices"
}
]]

local tableA = json.decode(snapshot1)
local tableB = json.decode(snapshot2)
if tableA.ActTime ~= tableB.ActTime then
    print("ActTime has changed from [ "..tableA.result[1].ActTime.." ] to [ "..tableB.result[1].ActTime.." ]") end
if tableA.ServerTime ~= tableB.ServerTime then
    print("ActTime has changed from [ "..tableA.ServerTime.." ] to [ "..tableB.ServerTime.." ]") end
if tableA.result[1].LastUpdate ~= tableB.result[1].LastUpdate then
    print("result:LastUpdate has changed from [ "..tableA.result[1].LastUpdate.." ] to [ "..tableB.result[1].LastUpdate.." ]") end

As always, many thanks to this community for their help..

>Solution :

Since these are JSON tables, we can make the following assumptions greatly simplifying the problem of comparing two tables:

1. There are no cycles, backreferences, duplicate references etc.; the object is a proper tree;
2. Tables are either dictionaries/objects (string keys) or arrays/lists (integer keys from 1 to n, possibly holes depending on how your JSON library handles nulls)
3. Values are either booleans, numbers, strings, or one of the aforementioned two table types

we can now write a recursive function that returns the difference in table form:

local function table_changes(original, modified)
    if original == modified then return nil end -- no changes
    if not (type(original) == "table" and type(modified) == "table") then return {from = original, to = modified} end -- primitive types
    local result = {}
    for k, v in pairs(original) do
        result[k] = table_changes(v, modified[k])
    end
    for k, v in pairs(modified) do
        if original[k] == nil then
            result[k] = table_changes(nil, v)
        end
    end
    return next(result) and result -- return nil for an empty table (no changes)
end

this will return a table like the following:

{
    ServerTime = {
        from = "2017-07-29 22:40:52",
        to = "2017-07-31 11:40:52"
    },
    Sunrise = {
        from = "05:50",
        to = "05:52"
    },
    Sunset = {
        from = "21:28",
        to = "21:29"
    },
    result = {
        {
            LastUpdate = {
                from = "2017-07-29 21:16:22",
                to = "2017-07-30 21:16:22"
            }
        }
    }
}