Skip to main content

Manage GCP bucket using gsutil command

Manage GCP bucket using gsutil command

How to create a GCP storage bucket using CLI.

Image credit `wikipedia` & Google cloud


In this tutorial, I will explain to you how to manage and perform basic tasks in GCP cloud storage using the command line. Cloud storage is worldwide and highly durable object-based storage. You can access data instantly at any time from storage class. To manage cloud storage using command line we need to configure gsutils

Install gsutil on Ubuntu 18.04.

Using gsutil you can manage GCP cloud storage. You can perform tasks related to cloud storage using gsutil command. So let's install gsutil on ubuntu.
There are different methods to install gsutil But here I will install it using python package Index pip.
  • Install the required packages.
sudo apt-get install gcc python-dev python-setuptools libffi-dev
  • Install the pip installer.
sudo apt-get install python-pip
  • Install gsutil using pip.
sudo pip install gsutil
Now, You are ready to use gsutil.

OR Gcloud-SDK.

You can also install cloud SDK to access and manage all the GCP cloud resources.
curl https://sdk.cloud.google.com | bash
exec -l $SHELL
gcloud init

Create cloud storage bucket.

Using the gsutil mb the command you can create a new bucket in GCP cloud storage. Let's start our first cloud storage bucket vishalvyas-bucket.
gsutil mb gs://vishalvyas-bucket/
Creating gs://vishalvyas-bucket/...
Our first bucket is created. Let's check the bucket.
Note : You can specify -l for the location of your bucket. like `ASIA-EAST1

List the bucket.

You can list the buckets using ls command.
gsutil ls
gs://vishalvyas-bucket/ You can see that bucket in GCP storage.

Upload content to the GCP bucket.

Now, Let' play with the bucket and upload some content on it. I have 2 files called 1.txt & 2.txt in my pc. I want to upload that files in the bucket which we have created. We will use cp to upload objects from local to storage bucket. Let's upload it.
gsutil cp 1.txt 2.txt gs://vishalvyas-bucket

Copying file://1.txt [Content-Type=text/plain]...
Copying file://2.txt [Content-Type=text/plain]...                               
/ [2 files][    0.0 B/    0.0 B]                                                
Operation completed over 2 objects.                                              
You can see that 2 objects have been uploaded. let's view it.
gsutil ls gs://vishalvyas-bucket/
gs://vishalvyas-bucket/1.txt
gs://vishalvyas-bucket/2.txt

You can see that object which we have uploaded.

Download object from GCP bucket to pc.

Let's download the data from the GCP bucket. I want to download 2.txt file from bucket.
gsutil cp gs://vishalvyas-bucket/2.txt .
Copying gs://vishalvyas-bucket/2.txt...
/ [1 files][    0.0 B/    0.0 B]                                                
Operation completed over 1 objects.  
You can find the download file in your working directory.

Make bucket objects publically accessible.

You can make your object publicly accessible using acl ch  command. So that all the users can use that object.
gsutil acl ch -u AllUsers:R gs://vishalvyas-bucket/2.txt
Updated ACL on gs://vishalvyas-bucket/2.txt
Now all the users can access 2.txt file. To remove the permission, Use the below command.
gsutil acl ch -d AllUsers gs://vishalvyas-bucket/2.txt
Updated ACL on gs://vishalvyas-bucket/2.txt
Now, Only you can access that file.

Give bucket access to the particular user

You can share your bucket object with the users you want. You want to just give their email ID and permission to the bucket.
gsutil iam ch user:shiva@companyid.com:objectCreator,objectViewer gs://vishalvyas-bucket
Now, User with the above email id will access and view your bucket object with the given permission.
You can remove the permission using the command below.
gsutil iam ch -d user:shiva@companyid.com:objectCreator,objectViewer gs://my-awesome-bucket

Now, We have removed the user access to this bucket.

Delete objects

Let's delete the bucket now. You can delete the bucket using rm -r command.
❯ gsutil rm -r gs://vishalvyas-bucket

Removing gs://vishalvyas-bucket/1.txt#1581571948280886...
Removing gs://vishalvyas-bucket/2.txt#1581571949191136...                       
/ [2 objects]                                                                   
Operation completed over 2 objects.                                              
Removing gs://vishalvyas-bucket/...
Bucket and its contents are deleted now, As You can see that everything is deleted now.
Hope this tutorial will help you to learn and manage GCP cloud storage.

Comments

Post a Comment

Popular posts from this blog

Another instance of Certbot is already running - Solved - Linux guru

Error : Another instance of Certbot is already runningcertbot --server https://acme-v02.api.letsencrypt.org/directory --manual --preferred-challenges dns --installer nginx -d *.vishalvyas.com Another instance of Certbot is already running Reason : when you run certbot form your machine and unexpectedly stop the command, Then cert bot is not running but it left some .certbot.lock files behind.You need to kill the certbot instance form your machine. Solution : Run the below command to find killed certbot.find / -type f -name ".certbot.lock" You can see result of the command, If there are, you can remove them. Run below command to remove them.find / -type f -name ".certbot.lock" -exec rm {} \; And try again.

Install Apache Using Ansible Playbooks

Ansible is a automation tool which is widely used, you can install and install, configure and manage number of system and services remotely. you can install software and manage services and tasks without needing manually log in to each servers. you have to install ansible in one machine and use ssh to communicate host each other.



Ansible uses Playbooks which is written in YAML format. it's uses module base format. with playbook can run multiple tasks at time and provide more advance functionality, YAML file always start with "---" syntax. In this article we will see how to install and configure apahce2 using ansible YAML script. Master server : 10.80.253.11 [Ansible server, ] Slave 1 : 10.80.253.12 [need to install apache2 in slave 1] Slave 2 : 10.80.253.13 [need to install apache2 in slave 2] We need to configure slave server info in our ansible configuration file, click here to know how to add client machine to ansible. Let's update our apache2.yml file and install…

Create aws s3 bucket using cli

In this article i will explain you to how to manage s3 bucket and objects using AWS cli command line interface. for that you will have to configure AWS cli interface to your local linux machine. You can configure cli from. HERE.




After configured aws cli to your local machine, you can run all below commands. How to create bucket using AWS Cli.root@master:~# aws s3 mb s3://linuxgurubucket make_bucket: linuxgurubucket Listing Bucket : To list all bucket : root@master:~# aws s3 ls 2018-05-01 15:28:37 linuxgurubucket To list all objects from bucket :root@master:~# aws s3 ls s3://linuxgurubucket 2018-05-01 15:32:16 71515 vishal.jpg How to copy local file to s3 using aws cp command :root@master:~# aws s3 cp ansible.sh s3://linuxgurubucket/ upload: ./ansible.sh to s3://linuxgurubucket/ansible.sh To check it's upload or not :root@master:~# aws s3 ls s3://linuxgurubucket/ 2018-05-01 15:34:19 82 ansible.sh 2018-05-01 15:32:16 71515 vishal.jpg To remove file from s3 bucket using cli: