How Can I Serve Temporary Files From Python Pyramid

Currently, I'm just serving files like this: # view callable def export(request): response = Response(content_type='application/csv') # use datetime in filename to avoid co

Solution 1:

You do not want to set a file pointer as the app_iter. This will cause the WSGI server to read the file line by line (same as for line in file), which is typically not the most efficient way to control a file upload (imagine one character per line). Pyramid's supported way of serving files is via pyramid.response.FileResponse. You can create one of these by passing a file object.

response = FileResponse('/some/path/to/a/file.txt')
response.headers['Content-Disposition'] = ...

Another option is to pass a file pointer to app_iter but wrap it in the pyramid.response.FileIter object, which will use a sane block size to avoid just reading the file line by line.

The WSGI specification has strict requirements that response iterators which contain a close method will be invoked at the end of the response. Thus setting response.app_iter = open(...) should not cause any memory leaks. Both FileResponse and FileIter also support a close method and will thus be cleaned up as expected.

As a minor update to this answer I thought I'd explain why FileResponse takes a file path and not a file pointer. The WSGI protocol provides servers an optional ability to provide an optimized mechanism for serving static files via environ['wsgi.file_wrapper']. FileResponse will automatically handle this if your WSGI server has provided that support. With this in mind, you find it to be a win to save your data to a tmpfile on a ramdisk and providing the FileResponse with the full path, instead of trying to pass a file pointer to FileIter.

http://docs.pylonsproject.org/projects/pyramid/en/1.4-branch/api/response.html#pyramid.response.FileResponse

Solution 2:

Update:

Please see Michael Merickel's answer for a better solution and explanation.

If you want to have the file deleted once response is returned, you can try the following:

import os
from datetime import datetime
from tempfile import NamedTemporaryFile

# view callabledefexport(request):
    response = Response(content_type='application/csv')
    with NamedTemporaryFile(prefix='XML_Export_%s' % datetime.now(),
                            suffix='.xml', delete=True) as f:
        # this is where I usually put stuff in the file
        response = FileResponse(os.path.abspath(f.name))
        response.headers['Content-Disposition'] = ("attachment; filename=Export.xml")
        return response

---

You can consider using NamedTemporaryFile:

NamedTemporaryFile(prefix='XML_Export_%s'% datetime.now(), suffix='.xml', delete=True)

Setting delete=True so that the file is deleted as soon as it is closed.

Now, with the help of with you can always have the guarantee that the file will be closed, and hence deleted:

from tempfile import NamedTemporaryFile
from datetime import datetime

# view callabledefexport(request):
    response = Response(content_type='application/csv')
    with NamedTemporaryFile(prefix='XML_Export_%s' % datetime.now(),
                            suffix='.xml', delete=True) as f:
        # this is where I usually put stuff in the file
        response.app_iter = f
        response.headers['Content-Disposition'] = ("attachment; filename=Export.xml")
        return response

Solution 3:

The combination of Michael and Kay's response works great under Linux/Mac but won't work under Windows (for auto-deletion). Windows doesn't like the fact that FileResponse tries to open the already open file (see description of NamedTemporaryFile).

I worked around this by creating a FileDecriptorResponse class which is essentially a copy of FileResponse, but takes the file descriptor of the open NamedTemporaryFile. Just replace the open with a seek(0) and all the path based calls (last_modified, content_length) with their fstat equivalents.

classFileDescriptorResponse(Response):
"""
A Response object that can be used to serve a static file from an open
file descriptor. This is essentially identical to Pyramid's FileResponse
but takes a file descriptor instead of a path as a workaround for auto-delete
not working with NamedTemporaryFile under Windows.

``file`` is a file descriptor for an open file.

``content_type``, if passed, is the content_type of the response.

``content_encoding``, if passed is the content_encoding of the response.
It's generally safe to leave this set to ``None`` if you're serving a
binary file.  This argument will be ignored if you don't also pass
``content-type``.
"""def__init__(self, file, content_type=None, content_encoding=None):
    super(FileDescriptorResponse, self).__init__(conditional_response=True)
    self.last_modified = fstat(file.fileno()).st_mtime
    if content_type isNone:
        content_type, content_encoding = mimetypes.guess_type(path,
                                                              strict=False)
    if content_type isNone:
        content_type = 'application/octet-stream'
    self.content_type = content_type
    self.content_encoding = content_encoding
    content_length = fstat(file.fileno()).st_size
    file.seek(0)
    app_iter = FileIter(file, _BLOCK_SIZE)
    self.app_iter = app_iter
    # assignment of content_length must come after assignment of app_iter
    self.content_length = content_length

Hope that's helpful.

Solution 4:

There is also repoze.filesafe which will take care of generating a temporary file for you, and delete it at the end. I use it for saving files uploaded to my server. Perhaps it can be useful to you too.

Solution 5:

Because your Object response is holding a file handle for the file '/temp/XML_Export_%s.xml'. Use del statement to delete handle 'response.app_iter'.

del response.app_iter