PubSub & BigQuery subscription using topic schema

With the new direct PubSub to BigQuery announced July 2022, we can now create BigQuery subscriptions that get data pushed from PubSub directly.

When it comes to timestamp fields, how are they mapped to BigQuery? Things I tried:

• Enable "use schema" in PubSub and create a BQ table with a created_at: TIMESTAMP

  MEDevel.com: Open-source for Healthcare and Education

  Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

  Visit Medevel

• The documentation doesn’t seem to specify a timestamp mapping for Protobuf, so ignoring this.

• When using Avro Logical fields, the error I’m getting when creating a subscription is like Incompatible schema type for field created_at: expected INT64, got TIMESTAMP.

Example of PubSub topic schemas tried:

{
  "type": "record",
  "name": "Avro",
  "fields": [
    {
      "name": "created_at",
      "type": {
        "type": "<long|int>",
        "logicalType": "timestamp-<millis|micros>"
      }
    }
  ]
}

What’s the correct way to create a PubSub schema for Timestamp fields?

>Solution :

Right now, logical types are not supported with BigQuery subscriptions. The documentation is being updated to reflect this fact and support is being added as we speak. Expect it to be ready within a few weeks. For now, the TIMESTAMP logical type would map to the underlying type, which would be INT64 in this case.