Setting up to Load and Move Data to Cloud Storage and Setting Up a Cluster Using the Command Line

Loading and moving data can be done in the command line using the gsutil command. To create a bucket, use the gsutil mb command; “mb” is short for “make a bucket.”

gsutil mb gs://[BUCKET_NAME]/

Keep in mind that bucket names must be globally unique.

To create a bucket named bucket1 , you would use the following command:

gsutil mb gs://bucket1/

To upload a file from your local device or a GCP virtual machine (VM), you can use the gsutil cp command to copy files.

The command is as follows:

gsutil cp [LOCAL_OBJECT_LOCATION] gs://[DESTINATION_BUCKET_NAME]/

For example, to copy a fi le called README.txt from /home/mydir to the bucket bucket1 , you’d execute the following command from your client device command line:

gsutil cp /home/mydir/README.txt gs://bucket1/

Similarly, if you’d like download a copy of your data from a Cloud Storage bucket to a directory on a VM, you could log into the VM using SSH and issue a command such as this:

gsutil cp gs://bucket1/README.txt /home/mydir/ In this example, the source object is on Cloud Storage, and the target file is on the VM from which you are running the command.

The gsutil tool has a move command; its structure is as follows:

gsutil mv gs://[SOURCE_BUCKET_NAME]/[SOURCE_OBJECT_NAME] \ gs://[DESTINATION_BUCKET_NAME]/[DESTINATION_OBJECT_NAME]

To move the README.txt file from bucket1 to bucket2 and keep the same filename, you’d use this command:

gsutil mv gs://bucket1/README.txt

gs://bucket2/

Setting Up a Cluster

kubectl and gcloud

The most important tool you use when setting up a Kubernetes environment is the kubectl command. This command allows you to interact with the Kubernetes API. It is used to create, update, and delete Kubernetes resources like pods, deployments, and load balancers.

Kubernetes doesn’t know or care where it is running, so there is no built in way for it to communicate with your chosen cloud provider to rent nodes on your behalf. Because we are using Google Kubernetes Engine for this tutorial, we will need to use the gcloud command for these tasks.

The gcloud tool provides the primary command-line interface for Google Cloud, and kubectl provides the primary command-line interface for running commands against Kubernetes clusters.

This tutorial assumes you already sign up for a GCP account, set up a project, enable billing, and install the command line tools.

Once you have your environment ready to go, you can create a cluster by running the following commands:

Along with the gcloud command, you can manage your resources through the Google Cloud Console page. After running the previous commands, you should see your cluster appear under the GKE section. You should also see a list of the VMs provisioned as your nodes under the GCE section. Note that although the GCE UI allows you to delete the VMs from this page, they are being managed by your cluster, which will re-create them when it notices they are missing. When you are finished with this tutorial and want to permanently remove the VMs, you can remove everything at once by deleting the cluster itself.

Stay tuned till next blog

Google Cloud @kubernets @basics

Written by

I am Google Cloud Certified Associate Cloud Engineer and MS Azure Certifed. I have been working on different fields including Data Science, DevOps and Anthos.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store