google app engine - How to catch BigQuery loading errors from an AppEngine pipeline -

I created a pipeline on the app engine that loads data from cloud storage to bigQuery. It works fine, unless there is an error. How can I load an exception by BigQuery from my AppEngine code?

The code in the pipeline looks like this:

  # run the work credentials = app auspicious credentials (scope = scope)) Http = credentials.authorize (httplib2.Http )) Bigquery_service = build ("bigquery", "v2", http = http) jobCollection = bigquery_service.jobs () results = jobCollection.insert (projectId = PROJECT_ID, body = build_job_data (table_name, cloud_storage_files)) when receiving status (all No) Do not run and no runtime. IS_Shooting_Down (): Try: Job = JobCalysis.Gate (Project ID = PROJECT_ID, Job ID = DART Response) .execute () # Something from Job. 'Status'): exc_type, exc_value, exc_traceback = sys.exc_info () logging. Terror (traceback.format_exception (exc_type, exc_value, exc_traceback) time.sleep (30)  

This gives me status error, or major connectivity errors, but what I see is Functional is Big Query can be Big Query when attempting to insert rows in table, like field format conversion errors, schema structure problems, or other problems.

If there is any "functional" error towards BigQuery, then this code will be executed successfully and normally, but no table will be written on BigQuery when it is so easy to debug You can use the HTTP error code from the exception ...

BigQuery is a REST API, so the returned response codes match the details of the HTTP error code.

Here are some codes that handle retrievable errors (connections, rate limits, etc.), but arises again This is an error type that does not expect it. HttpError, err: # If the error rate is the limit or connection error, then wait and try again # 403: Forbidden: Both Access Denied and Rate Limits # 408: Timeout # 500: Internal service error # 503: [403, 408, 500, 503] The service is missing on err.resp.status: Print '% s: retrievable error% s, wait'% (self.thread_id, Err. Resp.status,) time.sleep (5) Other: raise

If you need better error handling, BigqueryError in the Beech Command Line client Check the box (this can be available at code.google.com, but recently it is not possible with the switch, this is no more, but if you have gcloud installed, bq.py and bigquery_client.py files should be in the installation Should.)