Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

Write to bigquery using java cloud function

I want to create a table then write to bigquery using a cloud function, but I dont want to duplicate the data in the tables so I deleted the table first then create the table each time I call the function.

    public static void runCreateTable() {
        Schema schema =
                Schema.of(
                        Field.of("id", StandardSQLTypeName.INT64),
                        Field.of("name", StandardSQLTypeName.STRING));
        createTable(DATASET, TABLE_NAME, schema);
    }
public static void createTable(String datasetName, String tableName, Schema schema) {
        try {
            BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();
            TableId tableId = TableId.of(datasetName, tableName);
            TableDefinition tableDefinition = StandardTableDefinition.of(schema);
            TableInfo tableInfo = TableInfo.newBuilder(tableId, tableDefinition).build();
            bigquery.create(tableInfo);
            System.out.println("Table created successfully");
        } catch (BigQueryException e) {
            System.out.println("Table was not created. \n" + e.toString());
        }
    }
BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();

            if (bigquery.getTable(TableId.of(DATASET, TABLE_NAME)).delete()) {
                runCreateTable();
                TableRow row = new TableRow();

                for (Map.Entry<String, Object> entry : campaign.entrySet()) {
                    row.set("id", entry.getKey()).set("name", entry.getValue());
                    bigquery.insertAll(InsertAllRequest.newBuilder(bigquery.getTable(TableId.of(DATASET, TABLE_NAME))).addRow(row).build());
                }
            }

So the error is when I deleted the table first the insertAll cannot find the table when it is recreated to write to I got this error: Table abc.abc_names not found

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

>Solution :

I suggest you avoid recreation of the table instead you can delete its content:

DELETE FROM TABLE_NAME WHERE 1=1

Also, you can check the below answer:

"Not found: Table" for new bigquery table

Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading