Setting up to Load and Move Data to Cloud Storage and Setting Up a Cluster Using the Command Line
Loading and moving data can be done in the command line using the gsutil command. To create a bucket, use the gsutil mb command; “mb” is short for “make a bucket.”
gsutil mb gs://[BUCKET_NAME]/
Keep in mind that bucket names must be globally unique.
To create a bucket named bucket1 , you would use the following command:
gsutil mb gs://bucket1/
To upload a file from your local device or a GCP virtual machine (VM), you can use the gsutil cp command to copy files.
The command is as follows:
gsutil cp [LOCAL_OBJECT_LOCATION] gs://[DESTINATION_BUCKET_NAME]/
For example, to copy a fi le called README.txt from /home/mydir to the bucket bucket1 , you’d execute the following command from your client device command line:
gsutil cp /home/mydir/README.txt gs://bucket1/
Similarly, if you’d like download a copy of your data from a Cloud Storage bucket to a directory on a VM, you could log into the VM using SSH and issue a command such as this:
gsutil cp gs://bucket1/README.txt /home/mydir/ In this example, the source object is on Cloud Storage, and the target file is on the VM from which you are running the command.
The gsutil tool has a move command; its structure is as follows:
gsutil mv gs://[SOURCE_BUCKET_NAME]/[SOURCE_OBJECT_NAME] \ gs://[DESTINATION_BUCKET_NAME]/[DESTINATION_OBJECT_NAME]
To move the README.txt file from bucket1 to bucket2 and keep the same filename, you’d use this command:
gsutil mv gs://bucket1/README.txt
gs://bucket2/
Setting Up a Cluster
kubectl and gcloud
The most important tool you use when setting up a Kubernetes environment is the kubectl command. This command allows you to interact with the Kubernetes API. It is used to create, update, and delete Kubernetes resources like pods, deployments, and load balancers.
Kubernetes doesn’t know or care where it is running, so there is no built in way for it to communicate with your chosen cloud provider to rent nodes on your behalf. Because we are using Google Kubernetes Engine for this tutorial, we will need to use the gcloud command for these tasks.
The gcloud tool provides the primary command-line interface for Google Cloud, and kubectl provides the primary command-line interface for running commands against Kubernetes clusters.
This tutorial assumes you already sign up for a GCP account, set up a project, enable billing, and install the command line tools.
Once you have your environment ready to go, you can create a cluster by running the following commands:
# create the cluster# by default, 3 standard nodes are created for our clustergcloud container clusters create my-cluster --zone us-east1-a#get the credentials so we can manage it locally through kubectl# creating a cluster can take a few minutes to completegcloud container clusters get-credentials my-cluster \ --zone us-east1-a
Along with the gcloud command, you can manage your resources through the Google Cloud Console page. After running the previous commands, you should see your cluster appear under the GKE section. You should also see a list of the VMs provisioned as your nodes under the GCE section. Note that although the GCE UI allows you to delete the VMs from this page, they are being managed by your cluster, which will re-create them when it notices they are missing. When you are finished with this tutorial and want to permanently remove the VMs, you can remove everything at once by deleting the cluster itself.
Stay tuned till next blog
Google Cloud @kubernets @basics
