Integrate with Machine Learning APIs: Challenge Lab
In this article, we will go through the lab to Integrate with Machine Learning APIs: Challenge Lab. In the previous, lab you will get familiar with Introduction to APIs in Google, Extract, Analyze, and Translate Text from Images with the Cloud ML APIs, Classify Text with the Natural Language API, Detect Labels, Faces, and Landmarks in Images with the Cloud Vision API, Entity and Sentiment Analysis with the Natural Language API, and Awwvision: Cloud Vision API from a Kubernetes Cluster.
The challenge contains 5 required tasks
- Grant the service account admin privileges for BigQuery and Cloud Storage.
- Create and download a service account credentials file to provide Google Cloud credentials to a Python application.
- Modify a Python script to extract text from image files using the Google Cloud Vision API.
- Modify a Python script to translate text using the Google Translate API.
- Check which languages are in the extracted data by executing a BigQuery SQL query.
Challenge Scenario
You have been asked to develop a process to analyze sets of images of signage to extract and translate any text in the images. This extracted text information will be used to help classify the images as part of a machine learning project that will use this image dataset for model training and evaluation. The images all contain text, but the text may be in any language. The images are stored in a Cloud Storage bucket that has been provided for you.
You must use a Python script to process each of the image files by sending them to the Google Vision API to identify the text in the image. The text from each image must be saved back to files on Cloud Storage, with a separate file for the text from each image. If the text locale is not English (locale=’en’), you must then send the text to the Google Translate API to get an English translation for the original text. Once all of the images have been processed, the script must upload the results to a BigQuery table.
This diagram outlines the process:
The processed text data must then be written out to a pre-existing BigQuery table called image_text_detail in a dataset in your project called image_classification_dataset.
A colleague on your team had started to work on the code to process the images based on a Python script previously used to process a set of text files using the Natural Language API. Your colleague has been moved to a separate project and you must now complete the task.
Most of the work on the script has been completed and the version you have been given will access a storage bucket, and iterate over every image file it finds. However, the specific API calls that need to be made to find the text in each image and then send that text to the Translation API have not yet been implemented.
You have been given a copy of the work in progress Python script and a set of sample images in a Cloud Storage bucket that is named after your lab Project ID.
Your colleague identified the unfinished parts of the script and commented on the API calls that need to be made. There are three unfinished parts in the script that you must complete to make the correct Machine Learning API calls. All of them are preceded with a comment using the label # TBD:. The final line of code uploads the result data to BigQuery. In the script, this line is disabled by a comment character. When you are satisfied that the rest of the script is working, remove the comment character to enable the final line.
Before you work on the script, you must prepare your environment by creating a service account with the correct permissions and download the credential file for that account. Once you have the service account credentials, you can modify the Python script and use it to process the image files.
To complete the challenge, the original extracted text, locale, and translated text data for all of the images must be loaded into the BigQuery table called image_text_detail. The code to do this is in the script but you must remove the comment characters to enable the line of code at the end of the script.
Once you have successfully processed the image files using the updated Python script and uploaded to data to BigQuery, you must confirm that image data has been successfully processed by running the following Query in BigQuery:
SELECT locale,COUNT(locale) as lcount FROM image_classification_dataset.image_text_detail GROUP BY locale ORDER BY lcount DESCcontent_copyThis query will report the number of signs of each language type it has found in the set of sample images.
1.Configure a service account to access the Machine Learning APIs, BigQuery, and Cloud Storage
- In the Cloud Shell, create a new service account that provides credentials for the script using the following commands.
export PROJECT=qwiklabs-gcp-00-c1a4e49284be
gcloud iam service-accounts create my-account --display-name my-account
gcloud iam service-accounts create PROJECT- Then you have created the account, bind the BigQuery Admin and Cloud Storage Admin roles to the Service Account to provide the IAM permissions required to process files from Cloud Storage, and insert the result data into a BigQuery table.
gcloud projects add-iam-policy-binding $PROJECT --member=serviceAccount:my-account@$PROJECT.iam.gserviceaccount.com --role=roles/bigquery.admin
gcloud projects add-iam-policy-binding $PROJECT --member=serviceAccount:my-account@$PROJECT.iam.gserviceaccount.com --role=roles/storage.admin
2.Create and download a credential file for your Service Account
Run the following commands to download the JSON format IAM credentials file for the service account, and set up the name of the credentials file as an environment variable for the Python script.
gcloud iam service-accounts keys create key.json --iam-account=my-account@$PROJECT.iam.gserviceaccount.comexport GOOGLE_APPLICATION_CREDENTIALS=key.json
3: Modify the Python script to extract text from image files
- Run the following command to copy the file
analyze-images.pyfrom the Cloud Storage bucket into the Cloud Shell.
gsutil cp gs://$PROJECT/analyze-images.py .
- Now go to open editor -> analyze-image.py
- Copy Paste the following code.
- Save it.
- I said to code copy and paste the above code because there are three unfinished parts in the script that you must complete to make the correct Machine Learning API calls.
- All of them are preceded with a comment using the label
# TBD:. - You will need to make use of the Vision API, the Translation API, and the BigQuery API.
- The import of the Google Cloud Library modules for the required APIs has been done in line 7 of the script file.
- Note down the name of their API clients that have also been declared from lines 25 to 32.
4.Modify the Python script to translate the text using the Translation API
- Copy and paste the following code to the Cloud Shell.
python3 analyze-images.py $DEVSHELL_PROJECT_ID $DEVSHELL_PROJECT_ID5.Identify the most common non-English language used in the signs in the data set
- In the GCP Console go to Navigation Menu >BigQuery.
- Copy and paste the following code to the Query editor and then click Run.
SELECT locale,COUNT(locale) as lcount FROM image_classification_dataset.image_text_detail GROUP BY locale ORDER BY lcount DESCCongratulations! Done with the challenge lab.
If you Want to Connect with Me:
Linkedin: https://www.linkedin.com/in/mayank-chourasia-38421a134/
Twitter: https://twitter.com/ChourasiaMayank.
Thank you Stay Safe, Stay healthy.
Stay tuned till the next blog.
