Getting the overall number and filesize quota of all your items in an S3 bucket is more complicated than you’d think, especially if your bucket contains more than a thousand items.
How to
First you need to install the AWS-SDK gem.
gem install aws-sdk
With the AWS-SDK included, you can access your bucket and retrieve a list of objects inside it:
@aws_count = 0
@aws_size = 0
s3_object_count(s3_bucket_objects)
The counting itself, needs to be done recursively, as each call to list_objects_v2 will only return a maximum number of 1000 objects:
# Get back S3 object listing, starting at start_after
def s3_bucket_objects(start_after = nil)
s3_connection = Aws::S3::Client.new(
region: ENV["aws_region"],
credentials: Aws::Credentials.new(<your_aws_access_key_id>, <your_aws_secret_access_key>)
)
s3_connection.list_objects_v2(
bucket: <your_aws_bucket>,
start_after: start_after
)
end
# Recursively count objects from an S3 object listing
def s3_object_count(bucket_objects)
bucket_objects.contents.each do |object|
unless object.nil?
@aws_count += 1
@aws_size += object.size
end
end
return if bucket_objects.is_truncated == false
s3_object_count(s3_bucket_objects(bucket_objects.contents.last.key))
end
@aws_count and @aws_size now should reflect the number of objects and your filesize quota in bytes.
If you are using this with Rails, you can use the view helper number_to_human_size inside your views to output @aws_size in a more human friendly format, like ‘1.61 GB’.