encode_utils.aws_storage

class encode_utils.aws_storage.S3Upload(bucket_name, acl='public-read', key_path='')

Bases: object

Simplifies the process of uploading files to a bucket in a specific location with the specified acl.

Parameters:

• bucket_name – str. The name of the bucket to upload files to, i.e. pulsar-encode-assets.
• acl – str. See possible values at https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client.put_object.
• key_path – str. The directory path in the specified bucket to upload all files to.

upload(filename)

Returns:The key/path of the newly created object. Return type:str

class encode_utils.aws_storage.S3Object(bucket_name='', key='', s3_uri='')

Bases: object

Represents an object in a S3 bucket. Internally used for calculating the md5sum and file size when submitting files to the ENCODE Portal.

You must set the appropriate AWS keys as documented in the wiki.

Parameters:

• bucket_name – str. The name of the S3 bucket that contains your file of interest. For example, “mybucket”.
• key – str. The object path in the bucket. For example, /path/to/reads.fastq.gz.
• s3_uri – Fully qualified path to the object in the bucket, i.e. s3://pulsar-lims-assets/path/to/reads.fastq.gz. If this is set, then the buket_name and key parameters are ignored since they will be set internally by parsing the value of s3_uri.

md5sum()

Retrieves the ETag as the md5sum. As explained here, an object’s ETag may or may not be equal to its MD5 digest. In most cases, however, it will be. If ETags on your objects aren’t equal, then this method shouldn’t be used.

size()

Fetches the size of the S3 object by reading its content_length attribute.