where to buy misoprostol online how to buy valtrex
Cross-platform file locking support in Python | Evan Fosmark

Cross-platform file locking support in Python

On occasion, one requires the need to lock a file. Now, this is relatively easy if you’re targeting a specific platform because there is often a function in the library to do it for you. But what if you want to target a larger set of platforms? The following is a solution I wrote up today. It’s lockfile creation is an atomic operation and thus doesn’t suffer from any race conditions. It should work in both Windows and Unix environments.

# Copyright (c) 2009, Evan Fosmark
# All rights reserved.
# 
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met: 
# 
# 1. Redistributions of source code must retain the above copyright notice, this
#    list of conditions and the following disclaimer. 
# 2. Redistributions in binary form must reproduce the above copyright notice,
#    this list of conditions and the following disclaimer in the documentation
#    and/or other materials provided with the distribution. 
# 
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
# 
# The views and conclusions contained in the software and documentation are those
# of the authors and should not be interpreted as representing official policies, 
# either expressed or implied, of the FreeBSD Project.
 
import os
import time
import errno
 
class FileLockException(Exception):
    pass
 
class FileLock(object):
    """ A file locking mechanism that has context-manager support so 
        you can use it in a with statement. This should be relatively cross
        compatible as it doesn't rely on msvcrt or fcntl for the locking.
    """
 
    def __init__(self, file_name, timeout=10, delay=.05):
        """ Prepare the file locker. Specify the file to lock and optionally
            the maximum timeout and the delay between each attempt to lock.
        """
        self.is_locked = False
        self.lockfile = os.path.join(os.getcwd(), "%s.lock" % file_name)
        self.file_name = file_name
        self.timeout = timeout
        self.delay = delay
 
 
    def acquire(self):
        """ Acquire the lock, if possible. If the lock is in use, it check again
            every `wait` seconds. It does this until it either gets the lock or
            exceeds `timeout` number of seconds, in which case it throws 
            an exception.
        """
        start_time = time.time()
        while True:
            try:
                self.fd = os.open(self.lockfile, os.O_CREAT|os.O_EXCL|os.O_RDWR)
                break;
            except OSError as e:
                if e.errno != errno.EEXIST:
                    raise 
                if (time.time() - start_time) >= self.timeout:
                    raise FileLockException("Timeout occured.")
                time.sleep(self.delay)
        self.is_locked = True
 
 
    def release(self):
        """ Get rid of the lock by deleting the lockfile. 
            When working in a `with` statement, this gets automatically 
            called at the end.
        """
        if self.is_locked:
            os.close(self.fd)
            os.unlink(self.lockfile)
            self.is_locked = False
 
 
    def __enter__(self):
        """ Activated when used in the with statement. 
            Should automatically acquire a lock to be used in the with block.
        """
        if not self.is_locked:
            self.acquire()
        return self
 
 
    def __exit__(self, type, value, traceback):
        """ Activated at the end of the with statement.
            It automatically releases the lock if it isn't locked.
        """
        if self.is_locked:
            self.release()
 
 
    def __del__(self):
        """ Make sure that the FileLock instance doesn't leave a lockfile
            lying around.
        """
        self.release()

The above class is best used in a context manager fashion through the with statement like in the example below:

with FileLock("test.txt", timeout=2) as lock:
    print("Lock acquired.")
    # Do something with the locked file

The largest downside of this is that the directory the file is located in must be writable. I hope this code helps you. Of course, if you have a better recipe, please share it in the comments. ;)

 

 

25 Comments

  1. Ville Kaseva wrote,

    Is there bug in acquire() method? Can timeout occur without entering Except: ??

  2. Evan wrote,

    Ville,

    There shouldn’t be any bug in acquire(). It may seem that there would be a race condition, but the following line is actually atomic:

    os.open(self.lockfile, os.O_CREAT|os.O_EXCL|os.O_RDWR)

    A timeout hasn’t occured outside the Except block when I’ve used it, but if it has for you, I’d love to see the trackback.

  3. Pat wrote,

    Could you please indicate the license under which this cross-platform file-locking code is available? Thanks!

  4. Dan wrote,

    Thanks for the code. The only change I made was to use os.mkdir and os.rmdir instead of os.open and os.unlink.

  5. Evan wrote,

    Pat, the code is lisenced under the BSD license.

    Dan, so I’m guessing that you’re using a “lockfolder” instead of a lockfile? Be sure that is an atomic operation otherwise you may run into problems.

  6. John Giotta wrote,

    This is a very useful script. Now to make it work for my environments running 2.4

  7. Steve wrote,

    Evan,
    What would happen when the process dies but never unlocked the file? I am wanting to use your solution but need to make sure that this case is handled.

  8. Matt Chaput wrote,

    This (and directory creation, which is also atomic and IMHO makes for clearer code) will leave behind a stale lock file/directory if your code doesn’t delete it (e.g. if you get an error and the process quits), preventing anyone else from locking the resource until you clean it up manually.

  9. Vinay Sajip wrote,

    @John Giotta: There’s a simpler version of this which I wrote back in 2007, which works on older Pythons but has no bells/whistles/context manager. See here: http://code.activestate.com/recipes/519626-simple-file-based-mutex-for-very-basic-ipc/

    Note that it writes the PID of the locking process to the file, so you can see who’s got it (and this also allows cleanup – e.g. if there’s no such process).

  10. PP wrote,

    It works very well!
    Many thanks!
    PP

  11. Rick Harris wrote,

    Evan, thanks for posting this, was a big help. I ended up using your FileLock to build a file-based counting semaphore which is used by http://github.com/rconradharris/filecache.

    Great job!

  12. Suan wrote,

    Hi, I ended up using this in my LG TV serial control library project too: https://github.com/suan/libLGTV_serial

    Thanks for writing this!

  13. Bill Pollifrone wrote,

    Hello. I want just wanted to thank you for posting this code. I only had to modify to use a temporary directory for the lock files so that I was sure to be able to write to it. I used it to ensure one process that reads files created by another process would not collide.

  14. Volker wrote,

    For the record, the code is not safe for all NFS implementations. See the O_EXCL section of the “open” manpage.

  15. Yaroslav Halchenko wrote,

    Exactly what I was looking for… But “BSD license” is too vague of a definition (see http://en.wikipedia.org/wiki/BSD_license for available choices) — most probably you refer to 3-clause version… would you be so kind to add a header with the copyright holder statement and license verbatim?

    thanks in advance

  16. Harry wrote,

    Thanks for writing and sharing this!

  17. Ash wrote,

    Thanks Evan for this neat little class – just the one I was looking for.

    I have included the code in my project: https://github.com/Tuxdude/repobuddy/blob/dev/RepoBuddyUtils.py with some minor modifications giving credits to you and this webpage – hope that should be fine :)

  18. sergio wrote,

    In the LockFile() constructor:
    self.lockfile = os.path.join(os.getcwd(), “%s.lock” % file_name)

    I wondering what does os.getcwd() equal if the application is started via init.rd startup/shutdown and maybe not having a tty?

  19. Tim wrote,

    Does this actually work? Or may I am using it wrong.

    I am on OS-X, I locked a file and then open it in both textwrangler and openoffice without issues, made changes, saved the file without any issues. But this Filelock class claims the file is locked.

    I want to properly lock a file so when my python script runs now one can save any changes to that file (and preferably not open it) until my python script ends.
    the file is designed for both manual editing in a text editor, or via a GUI I made, I dont want people doing both simultaneously because they will loose their work.

  20. Bryce wrote,

    Tim – that’s because this solution doesn’t use the OS file-locking calls. To make it platform independent, it just writes the a second file that signals to python that the file is locked. If you run a second python program that uses this class & tries to open a file, only then will the locking mechanism work.

  21. Stuart wrote,

    This is just what I needed. Thanks! I fixed a multithreading bug and added a few enhancements. Updated version is here:
    http://github.com/ilastik/lazyflow/blob/master/lazyflow/utility/fileLock.py

  22. Manish wrote,

    Is there any copywrite issue?
    I am hoping to use this in my project.
    If there is any copywrite thing is there. Do tell me.

  23. Evan wrote,

    I’ve specified more verbose information about copyright. It’s under a 2-clause BSD license. :)

  24. Jaakko wrote,

    Someone had copied this to github, so I forked and edited that. Here is my fork https://github.com/raphendyr/FileLock

    It handles path expansion correctly and also inserts pid into lockfile. I’m still working on howto check if lock is orphand (in a secure way).

  25. Sachin Nikam wrote,

    Hi,

    can we use above solution with multiple processes accessing single file and make sure that only one process write/read to that file.

Leave a comment