English ▼
Skip to page content Skip to main navigation
e.g. "templates" or "datastore"

# Google Cloud Storage API Functions (Python)

#### Experimental!

Google Cloud Storage API for Google App Engine is an experimental, innovative, and rapidly changing new feature for App Engine. Unfortunately, being on the bleeding edge means that we may make backwards-incompatible changes to Google Cloud Storage API for Google App Engine. We will inform the community when this feature is no longer experimental.

This document discusses the functions that the `google.appengine.api.files` package provides for the Google Cloud Storage Python API.

## Contents

Functions

- files.gs.create()
- files.open()
- files.File.write()
- files.File.read()
- files.File.tell()
- files.File.seek()
- files.File.close()
- files.finalize()

Classes

- BufferedFile()

## Functions

files.gs.create(filename, mime_type='application/octet', acl=None, cache_control=None, content_encoding=None, content_disposition=None, content_encoding=None, user_metadata=None)

  Create a Google Cloud Storage object in the specified bucket.

  Arguments:

  filename (Required)
    The Google Cloud Storage file name for the object, in the format `/gs/bucket/object_name`. The `object_name` can contain any sequence of Unicode characters, but cannot exceed 1024 bytes when UTF-8 encoded. Avoid using control characters that are considered illegal in XML 1.0 when creating an object name, as Google Cloud Storage omits these characters when listing objects. If an object with the same name already exists, the object created by this function will overwrite the existing object.
  mime_type='application/octet-stream' (Optional)
    The MIME type of the object. If you do not specify a MIME type, the default application/octet-stream MIME type will be set for you.
  acl=none (Optional)

    Set a predefined ACL on your object.

    If you do not set this parameter, the API sets this parameter as null and uses the default object ACL for that bucket (by default, this is `project-private`).

  cache_control=None (Optional)

    Set a Cache-Control header on your object. If you do not set this header, the Google Cloud Storage API sets:

    - `Cache-Control: public, max-age=3600` for publicly readable objects (objects with the ACL `public-read`)
    - `Cache-Control: no-cache, no-store, max-age=0, must-revalidate` for private objects (objects with the ACL `private`).
  content_encoding=None (Optional)

    If your object is compressed, specify the compression method using the Content-Encoding header.

  content_disposition=None (Optional)

    Set the Content-Disposition header for your object.

  user_metadata=None (Optional)

    Set custom headers and values using a Dictionary data set. These headers are served on `GET` requests to the user in the format x-goog-meta-custom_header: custom_value. For example, you can set custom headers as follows:

        params = {'author':'john', 'task':'1'}
        gs.create(... user_metadata=params)

    The headers are served on `GET` requests:

        HTTP/1.1 200 OK
        Cache-Control: my cache control
        ETag: "7d677bb83bf43de498557f4a47570b21"
        Content-Type: application/octet-stream
        Last-Modified: Tue, 13 Sep 2011 17:57:04 GMT
        x-goog-meta-author: john
        x-goog-meta-task: 1
        Content-Length: 45

  Returns

  Returns a handle to the newly created object.

Back to top

files.open(file, mode)

  Open an object for read or write, which returns a writable or readable `File` object. You must open a file for read or write before you can read or write to it. If you do not specify a mode, the default mode is `'r'` or read mode.

  You cannot open and write to a file that has already been finalized.

  Arguments:

  file (Required)
    The file to open, in the format of `/gs/bucket/object`.
  mode (Optional)
    Pass in 'a' to open a file for write and pass in 'r' to open a file for read. For example, the following snippet opens an object and reads from it:

        with files.open('/gs/mybucket/myobject/', 'r') as f:
          data = f.read(1)
          while data != "":
            print data
            data = f.read(1)
          print 'Done reading file!'

  Returns

  Returns a `File` class that has `read()`, `write()`, and `close()` methods.

files.File.write(data)

  Write to an object. Your object must have been created but not been finalized. You must open the object with `files.open()` before you can write to it.

  If you are writing to an unfinalized, existent object, `files.Files.write()` appends to the existing data.

  Arguments:

  data (Required)
    The data to be written to the file. For example, the following snippet creates a new object and writes to it:

        my_file = gs.create(file_name = '/gs/mybucket/myfile')
        with files.open(my_file, 'a') as f:
          f.write('Hello World!')
          f.write('Hello World Again!')

  Returns

  This function does not return any values.

Back to top

files.File.read(size)

  Reads up to the specified number of bytes of an object.

  Arguments:

  size (Required)

    The number of bytes to read, as an integer. The maximum number of bytes for one API call is 32MB.

  Returns

  Returns the data converted to a string. The string length is 0 when the end of the file is reached.

Back to top

files.File.tell()

  Give a file's current position. Only valid when the file is opened for read.

  Returns

  Returns a file's current position as an integer.

Back to top

files.File.seek(offset, whence=os.SEEK_SET)

  Set a file's current position.

  Arguments:

  offset (Required)
    The position to set the file as an integer.
  whence=os.SEEK_SET (Optional)

    The seek mode. By default, `os.SEEK_SET` is used, which sets the file's position using absolute seek. Alternatively, you can use `os.SEEK_CUR` to set the file's position relative to the current position.

  Returns

  This function does not return any values.

Back to top

files.File.close(finalize=false)

  Closes a file. Closing a file does not finalize it unless you explicitly set the finalize argument. You can reopen a file that has been closed to append to it. Until you finalize a file, you cannot read from it and once you finalize a file, you can no longer append to it.

  Arguments:

  finalize=false (Optional)

    By default, your file is not finalized when you close it. If you want to close and finalize your file, you can do so by passing in a true argument.

  Returns

  This function does not return any values.

Back to top

files.finalize(filename)

  Closes and finalizes an object. Once an object is finalized, you cannot write to it. You must finalize an object before it can be read.

  If an object has already been finalized, this call will be ignored. If the object has not been finalized, this call will close and finalize it.

  Arguments:

  filename (Required)

    The object from `files.create()` that has not been finalized. Note that you cannot pass an object from the `files.open()` method. To do so, you must call `files.File.close()`. For example:

        path = files.gs.create('/gs/mybucket/object')
        fp = files.open(path)
        fp.close()
        files.finalize(fp) # This does not work!
        files.finalize(path) # This works!

    You can also finalize an object using the handle from the `files.open()` method:

        fp = files.open(path, 'a')
        fp.close(true)

  Returns

  This function does not return any values.

Back to top

## Classes

BufferedFile(filename)

  `BufferedFile()` is a file-like object that reads the underlying file in chunks. You can use to this method to read from objects as an alternative to `files.File.read()`, with some notable differences:

  - `BufferedFile()` does not need to be closed
  - `BufferedFile()` does not need to use a `with` statement to read the file
  - `BufferedFile.read(n)` reads exactly `n` bytes unless the end of the file is reached

  For example, instead of using `files.File.read()`:

      with files.open('/gs/mybucket/object') as fp:
        fp.read(1000)

  You can use `BufferedFile()` instead:

      fp = files.BufferedFile('/gs/mybucket/object')
      fp.read(1000)

  Arguments:

  filename (Required)

    The file to read.

  Returns

  A `BufferedFile()` object with `seek()`, `tell()`, and `read()` methods.

files.BufferedFile.read(size)

  Read the underlying contents of the `BufferedFile()` object.

  Arguments:

  size (Required)

    The number of bytes to read as an integer. The actual number of bytes read is always equal to the size unless the end of the file is reached.

  Returns

  Returns the data converted to a string. The string length is 0 when the end of the file is reached.

files.BufferedFile.seek(offset, whence=os.SEEK_SET)

This function is the same as the files.File.seek() function.

files.BufferedFile.tell()

This function is the same as the files.File.tell() function.

Back to top