Questions tagged [google-cloud-storage]
Google Cloud Storage, also called GCS, is a cloud service for storing and accessing arbitrarily large objects.
162
questions
29
votes
2
answers
39k
views
Using JSON keys with google cloud gsutil
I have a ssh private key in a key.json file, and I want to use this credential to access a storage bucket using gsutil.
I can't seem to find anything about how to include json keys as a ...
19
votes
5
answers
42k
views
Google Cloud : check disk space usage
How to check the used disk space of Google Persistent Disk without looking into each of them?
17
votes
2
answers
54k
views
Service account does not have storage.buckets.get access to bucket
I'm trying to get a list of buckets in a project, using python like this:
from google.cloud import storage
storage_client = storage.Client(project='[project-id]')
bucket = storage_client.get_bucket([...
14
votes
2
answers
18k
views
Backing up Google Cloud Storage Bucket
Is there a way I can regularly backup one bucket into another one so that in case files get deleted from one bucket, they can always be recovered from another?
11
votes
2
answers
20k
views
permission errors using gcsfuse as nonroot user on google compute instance
We are currently trying to demo the google cloud platform for some workloads that require lots of storage. We are trying to use google cloud storage but are having problems mounting the buckets using ...
10
votes
2
answers
32k
views
How do I access a google cloud storage bucket using a service account from the command line?
I thought it would be pretty straight forward to do this, but I can't get it to work:
I'm trying to push files from a server (GCE) to a google cloud storage bucket. To avoid granting the gsutil ...
10
votes
1
answer
14k
views
Can I mount Google cloud storage bucket to a pod as a persistent disk. If yes how?
I have a gke cluster with nodes in 3 different Zones but same region. I want to use persistent disks to my pods. But the pods are created randomly in all the 3 zones.How can I use a gcePD (common for ...
9
votes
2
answers
6k
views
How to prevent DDoS attack on Google Cloud Storage
How can I protect my public images hosted on GCS from ddos attacks?
Does Google provide any protection, or should I pay for the bandwidth used by that attack?
- 93
8
votes
1
answer
3k
views
Lost Permissions to Google Cloud Storage Bucket
I accidentally deleted the "Storage Legacy Bucket Owner" permission on a bucket and now even though I'm the project owner and I created the bucket I cannot access it at all. What recourse do I have?
...
- 1,573
8
votes
4
answers
25k
views
Why can't I access a cloud storage object despite being a project owner?
I'm a project owner for a particular project, which has a bucket with a few hundred gigs of objects. I have recently discovered by a failed gcloud rsync that there are some objects in my bucket that I ...
- 183
8
votes
2
answers
2k
views
Keeping Google Cloud Storage buckets backed up
Basically, my question is: what are the best ways to keep my Google Cloud Storage bucket data safe?
I understand that using Object Versioning is a first measure you can take to protect yourself ...
7
votes
3
answers
6k
views
GCP: No access to Container Registry from Compute Engine
I have a Docker image pushed to Google Container Registry at gcr.io/my-project/my-image:latest. In the same project, I created a Google Compute Engine instance based on COS (Container-Optimized OS) ...
6
votes
1
answer
16k
views
Checking file's existence by name in a GCS bucket with Node.js
I'd like to know if Google Cloud Storage provides an efficient way of checking if a file with a given name exists in a given bucket with node.js. A method such as bucket.fileExists('someName') would ...
6
votes
1
answer
4k
views
Slow synchronisation stage on gsutil rsync?
I've just started to use GCS as backup for my web servers. One server has 1.2 million JPEGS (3.5TB) and this all rsynced over flawlessly over 10 hours or so.
The other has 2.5 million JPEGS (just ...
6
votes
2
answers
20k
views
How to get a download URL for files in Google Cloud Storage?
I have created a service using :
storage.Client(project='project_name', credentials=credentials)
https://googleapis.dev/python/storage/latest/client.html
And I'm able to upload, delete and list the ...
4
votes
2
answers
5k
views
getSignedUrl giving "SigningError: Failure from metadata server"
I'm using Google Cloud Functions with an HTTP trigger to create an endpoint for an API call. This endpoint is using the getSignedUrl function from the Cloud Storage package. I wanted to return a ...
4
votes
2
answers
4k
views
Cloud Run service account does not have permission to sign
I'm working on a program that will run on Google Cloud Run and has files stored in Google Cloud storage.
The problem I'm experiencing happens when attempting to generate a signed URL to download a ...
4
votes
1
answer
171
views
shutting down gcloud service in case of DDos attacks
my company has a small and nice product on google could (using public webapp, functions and storage).
it's been hours I'm searching for a solution like a kill switch for services in case of any threat ...
3
votes
1
answer
3k
views
How to handle retention policies in GCP Container Registry
GCP Container Registry is built on top of GCP Cloud Storage and one can set quite complex retention policies there.
However there's no such functionality in Container registry and as there's no ...
3
votes
1
answer
636
views
gsutil rsync "too many values to unpack" error
I'm trying to sync a folder with over 3k files to a bucket on Google Cloud with gsutil. The problem is that less than half of the files are uploaded and then I get this error:
too many values to ...
3
votes
1
answer
2k
views
Secure static web page hosted on Google Cloud Storage
I would like to add basic authentication to the website that hosts on Google Cloud Storage.
The site is connected to a load balancer, has a working SSL certificate and CDN.
Basic authentication ...
3
votes
1
answer
1k
views
Does gcsfuse sync all data to the system it is mounted on?
The scenario:
I have mounted a gcsfuse drive to my docker container so I can store and read data from Google Cloud Storage
The docker container only has a few gigs of storage, but I might have ...
3
votes
1
answer
3k
views
How to use JSON keys with google cloud gsutil to manage multiple Keys
We have multiple GCP Service account keys from different environment such as DEV,STAGING,..PROD.
I would like to run a command in my jenkins box which is there in Dev environment to create access DEV ...
3
votes
1
answer
3k
views
An internal error occurred while creating a Google Cloud Storage bucket
I have started my google cloud setup today.
Tried to follow instructions in interactive tutorial.
Setting up my First App that uses Node.js worked fine. But deploying app to Compute Engine failed with ...
3
votes
1
answer
6k
views
Copy list of files (-I flag) with gsutil preserving path
I am trying to copy all pictures and static files to a bucket of mine in Google Cloud Platform.
I am attempting this command from the root dir of my app:
find -regextype posix-extended -iregex ".*\.(...
- 435
3
votes
2
answers
2k
views
Configure a bucket's lifecycle policy like a recycle bin, keeping deleted files for a short while
I'm storing documents in GCS and I'd like to replicate the Windows/OSX notion of files moving to a Recycle Bin/Trash upon deletion so they can be recovered, rather than outright deleting them. In ...
3
votes
3
answers
3k
views
googleapi: Error 403: [email protected] does not have storage.objects.get access to the Google Cloud Storage object
I stay trying my project based on this: https://github.com/GoogleCloudPlatform/iot-smart-home-cloud
My issue is:
cloudfunctions.googleapis.com
google.cloud.functions.v1.CloudFunctionsService....
3
votes
1
answer
2k
views
Restrict Access to google cloud buckets by domain name
I want to host files that my users upload to my web application in google cloud buckets...the problem there is that I need to have the permissions open to the public because I also need to retrieve ...
3
votes
2
answers
632
views
Is there a way to mount my Google Cloud storage account on Windows Server 2016?
I want to be able to host site images on Google Cloud Storage. One way I am thinking to get it does is by adding a drive on my windows server that point to the cloud. Then my site would read the ...
3
votes
2
answers
2k
views
While accessing Fuse mounted Storage bucket, its showing 403 Forbidden Error
I have a google cloud project.
In that project. I have a VM instance.
I mounted uploads folder with Google Cloud Storage bucket using fuse.
While mounting I get an error like this,
Using mount point:...
- 131
3
votes
1
answer
6k
views
Exporting GCP Compute Engine instance fails: Quota 'SSD_TOTAL_GB' exceeded
I'm following instruction on: https://cloud.google.com/compute/docs/images/export-image
When running:
gcloud compute images export --destination-uri gs://my-bucket/my-image.tar.gz \
--image my-image ...
3
votes
1
answer
4k
views
Giving access on google cloud storage to users outside Google Apps domain
Scenario:
Registered domain xyz.com with Google Apps for Business
Created user [email protected], using which created a project on Google Cloud Platform
Created a bucket on Google Cloud Storage
Under ...
2
votes
2
answers
3k
views
How to store wordpress media files in Google Cloud Storage buckets?
I am using Google Cloud platform for my website hosting. My website mostly contains images.
I am using the persistent disc of size 50gb for storage purpose. If I increase the size of persistent disks,...
2
votes
2
answers
10k
views
NGINX proxy to GCS bucket with redirect all urls to index.html, getting 200 blank responses on nested routes
I am configuring a reverse-proxy from NGINX to a GCP Cloud Storage bucket containing static HTML, JS, image files, with a rewrite for all non-matching URLS to index.html since it is a single-page-...
2
votes
1
answer
2k
views
Google cloud storage public object cached on the server side
I have a Google bucket with read permission for allUsers, but it's not configured as a website (and no archiving). I experience an object caching even when I send requests with no-cache control:
...
- 143
2
votes
1
answer
451
views
Restrict Viewer Access to Cloud Storage Objects via ACL
I'm hosting a static site in a Cloud Storage Bucket with fine-grained permissions set. I'm attempting to set by-object ACL permissions so users can only access specific resources if they have a ...
2
votes
1
answer
2k
views
Bad credentials for bucket
I'm getting Bad credentials for bucket trying to mount a GCS bucket using gcsfuse. What I've done:
Created a bucket "mac-gcs-fuse"
Created a service account in the same project with all storage ...
2
votes
1
answer
817
views
How to limit budget for public AWS S3 or Google Cloud Storage? [duplicate]
I would like to upload public images for a web app to a service like AWS S3 or Google Cloud Plateform. This is going to be public images like user profile picture, homepage images... so there is not ...
2
votes
1
answer
3k
views
Why doesn't gcsfuse return to the command prompt after mounting?
I've successfully installed gcsfuse. When I run it, it prints File system has been successfully mounted but I don't get back to the shell prompt. (See the transcript below.) What am I doing wrong?
...
2
votes
1
answer
1k
views
service account with Storage Admin role does not have storage.buckets.get access
I'm trying to get a bucket on gcp from a pod on kubernetes using python.
Now the service account on that pod has a role of storage admin but for some reason I still get an exception when trying to ...
2
votes
2
answers
811
views
How do you assign storage permissions to a group of GCP service accounts?
How does one assign Google Cloud Storage bucket permissions to a group of users?
There's no bucket-level permissions that can be specified in roles, and there's no way to create a group as far as I ...
2
votes
3
answers
5k
views
Can't write any file into bucket only read after gcfuse
I deployed an instance of Wowza Streaming Engine on Google Cloud thank Made a bucket in Google Cloud Storage and mounted it all with GCFUSE. My bucket connected with success and I can see in it and ...
2
votes
1
answer
1k
views
Google Storage ACL - write only for public, but don't allow read, list, delete or update operations
I need to setup Google Storage access for 3rd parties (public) to allow them to upload one or more files, but they shouldn't be allowed to read, list, delete or update an existing file. This structure ...
- 21
2
votes
1
answer
1k
views
GCP LB with static website and kubernetes
I am trying to migrate an application from AWS to Googles container engine.
I have an S3 website (static files) living at my.domain.example and a backend living at the same url, but at the /api path. ...
2
votes
1
answer
2k
views
google-cloud storage bucket mounted to docker service
I am trying to achieve the following:
I want to mount a google-storage bucket to my docker swarm workers locally as a file-system. All swarm-workers have the necessary access-rights to the google ...
- 41
2
votes
1
answer
396
views
gsutil cp error: cannot have start index greater than total size
I'm trying to download a lot of images from a Google Bucket that my employer's web host (not so) helpfully set up. I run the command gsutil cp -r gs://bucket-link-info/images/ . and the download ...
2
votes
0
answers
390
views
gsutil rsync fails on Ubuntu with UnicodeDecodeError
I'm trying to rsync between google cloud storage and amazon s3. On my local machine (MacOS) this command works
gsutil rsync -d -r gs://mira-internal/data/exports/20191108 s3://mira-temp/raw/20191108
...
- 71
1
vote
3
answers
21k
views
Google Cloud Storage mount on windows
I use Google Cloud Storage successfully in Linux for backups. I first mount the storage and then easily copy the files.
Now I want to do the same in windows, but I can't find a way to mount Google ...
- 111
1
vote
2
answers
1k
views
Parallel downloads in Google Cloud Storage
Does anyone have experience with Google Cloud Storage? Is it made for a potential 10K of parallel requests to download 10-70mb files? Or would that require a CDN?
- 21
1
vote
3
answers
9k
views
Recover deleted folder in Google Cloud Storage
I have accidentally deleted one folder from the Google Cloud Storage.
How can I recover this folder?
Related Tags
google-cloud-platform × 121
gcsfuse × 17
gsutil × 11
google-iam × 8
kubernetes × 5
python × 5
gcloud × 5
fuse × 5
docker × 3
storage × 3
google-cloud-sql × 3
google-cloud-cdn × 3
windows × 2
security × 2
backup × 2
permissions × 2
performance × 2
wordpress × 2
deployment × 2
mount × 2
amazon-s3 × 2
more related tags
Hot Network Questions
- What happens when an anti-electron collides with a neutrino?
- What was the first work that used an earworm as a defense against a mental attack or probe?
- May a juror refuse to answer questions?
- Refusing to swear the oath by asking for a lawyer
- Does becoming a warlock mean said warlock will become a "Soul Coin" upon death?
- 'Cat' has a final 't'. — The letter "t" is the only one in the word "cat". Then why is it possible to use "a" here?
- Pronouns of professors in Statement of Purpose
- How to project a vertex to the nearest point along a vector?
- Can I use p-values without hypothesis?
- Does Eindhoven Airport offer flight transfer?
- How bad is topping out a suspension fork?
- Find a simple proof that π is irrational
- Does one accepted false statement allow proving anything?
- 220v Washer Dryer Machine Voltage Tolerance
- Why does the typical magneto switch have R on the left and L on the right?
- ST_ReducePrecision function does not work properly
- Parse nested absolute values
- How does one express the ordering of thoughts?
- Making a 3 x 8 grid with tetrominoes
- What would happen if a small black hole fell into a star?
- Chain drops when shifting up a gear at the front?
- Does kanji have meaning on their own, or they inherit their meaning(s) from the words they used in?
- Why doesn't the UK government end "bona vacantia"?
- Is it "unscientific" to be sceptical without offering alternative explanations?