Convert Unix Epoch to dateTime opensearch

June 6, 2023

I have a json log data with a field contain an Unxi Epoch in second and I want to convert it to humand readable pattern like dd/MMM/yyyy:HH:mm:ss.

I try to use data prepper of OpenSearch but i didnt found the Unix Epoch pattern to use it.

My json:

{"ua": "Mozilla/5.0%20(X11;%20Linux%20x86_64)%20AppleWebKit/537.36%20(KHTML,%20like%20Gecko)%20SamsungBrowser/21.0%20Chrome/110.0.5481.154%20Safari/537.36", "customField": "1685959346", "key3": "test", "securityRule": "test|112122|super"}

How can achieve this ?

>Solution :

You can use ingest pipeline for it. During ingestion, elasticsearch/opensearch will convert and enrich the data.

PUT _ingest/pipeline/epoch_conversion_pipeline
{
  "description": "Convert Unix Epoch to human-readable format",
  "processors": [
    {
      "set": {
        "field": "org_customField",
        "value": "{{customField}}"
      }
    },
    {
      "script": {
        "source": """
          ctx.customField = new SimpleDateFormat("dd/MMM/yyyy HH:mm:ss").format(new Date(Long.parseLong(ctx.org_customField) * 1000));
        """
      }
    }
  ]
}

PUT test_epoch_to_human
{
  "settings": {
    "default_pipeline": "epoch_conversion_pipeline"
  }
}

POST test_epoch_to_human/_doc/1
{
  "ua": "Mozilla/5.0%20(X11;%20Linux%20x86_64)%20AppleWebKit/537.36%20(KHTML,%20like%20Gecko)%20SamsungBrowser/21.0%20Chrome/110.0.5481.154%20Safari/537.36",
  "customField": "1685959346",
  "key3": "test",
  "securityRule": "test|112122|super"
}