.NET file locks

[ASB]

Scenario:-

Process 1 writes a file to a temp file. When it has done this it moves it to a directory "pending". Obviously the move gives an exclusive lock.

Process 2 is monitoring "pending" to look for files to process. It then process them. This is a separate thread which goes to sleep.

I have no control over process 1.

Now, process 2 will occasionally fail to process a file (not too important since it automatically tries again after a short interval). This occurs because the file locked by process 1. Obviously this is going to be the case whilst the file is being moved and before the OS has released the handle.

Of course the file can be locked at any time, it's being deserialised from XML so is initially passed to a stream reader (this throws the exception).

There isn't a time-out or any such constructor on a stream reader. Any suggestions. Not too bothered, but it's just a little messy.

[Zippy]

If process 2 re-tries, it sounds like it's behaving correctly. If it ain't broke ...

[AtW]

File move is a pretty atomic process (at least on Windows/NTFS) - sounds like process 1 isn't moving it after file handle is closed, but maybe even writing it there.

Anyway you don't have control over process 1, so it does not help you much whatever is the problem there, but you can move that file yourself in .NET and if that move is successful then file wasn't lock. This is pretty dirty, but much less so then trying to open it and fail.

If you dig win32 you should be able to get kernel .DLL call to check lock status - I think this might make you mark your .net code as unsafe though.

[Zippy]

File move is a pretty atomic process (at least on Windows/NTFS) - sounds like process 1 isn't moving it after file handle is closed, but maybe even writing it there.

Anyway you don't have control over process 1, so it does not help you much whatever is the problem there, but you can move that file yourself in .NET and if that move is successful then file wasn't lock. This is pretty dirty, but much less so then trying to open it and fail.

If you dig win32 you should be able to get kernel .DLL call to check lock status - I think this might make you mark your .net code as unsafe though.

I think the problem is that process 2 is trying to access the file during the move/copy process, so the lock is still on.

[ASB]

I think the problem is that process 2 is trying to access the file during the move/copy process, so the lock is still on.

Correct. It's actually doing a move not a copy so it is at least pretty quick (in any event the files are pretty small). The exceptions are occasional and it just produces an IO exception so it does a quick sleep and then retries. It's just that exceptions are expensive and I would prefer to be able to tell more precisely that it failed because of a lock error.

It's just that it's not really an exception - in that it is a predictable fail condition.

[mudskipper]

Quick search indicates there isn't - just handling the exception, which it sounds like you're doing. The reasons make sense - between checking for the lock and opening the file, it could become locked (in theory).

Can you check the timestamp of the file, and only process if > 5 secs old for example?

[AtW]

Is this "moving" by process 1 done on the same partition? If not, then it's actually copy/delete_old operation which would indeed have locks, normal moves should be instant and pretty much atomic on NTFS - at least in my experience: SKA does move a fair few files with other processes reading them.

Even if it's same partition it may well be that programmer of process 1 did not use proper move command and just did copy/delete.

Check this out, it may help - http://vbnet.mvps.org/code/fileapi/createfile_inuse.htm

[VectraMan]

normal moves should be instant and pretty much atomic on NTFS

Would you want to rely on that? Move it to a different system, with different partitions/drives, or have process 1 and 2 on different physical machines and it won't work anymore.

Seems to me detecting the failure, waiting, and having another go is the right approach. If the move is atomic, it won't ever fail, so you don't lose anything. If you need something more instantaneous, then you need to find a better mechanism for communicating between two processes than the existance of a file.

[AtW]

Would you want to rely on that? Move it to a different system, with different partitions/drives, or have process 1 and 2 on different physical machines and it won't work anymore.

It will work so long as the following principle applies: movement of file (which is a simple atomic rename operation in this case) should be within the same directory where it was created (say with .tmp extension) and written. If file is moved between partitions then it should be copied into temporary file and then renamed at the target disk - I am pretty sure in NTFS (at least recent versions) renaming (or moving within same disk) is atomic.

Is it perfect? No - but easy and pretty reliable so long as you use good file system like NTFS.

Either way in this case process 1 can't be changed, there ought to be a function in win32 to check if file is already open or some other lock is obtained on it.

[ASB]

Quick search indicates there isn't - just handling the exception, which it sounds like you're doing. The reasons make sense - between checking for the lock and opening the file, it could become locked (in theory).

Can you check the timestamp of the file, and only process if > 5 secs old for example?

Had done a lot of searching, and I came to the same conclusion. I did consider the timestamp, but the problems with that are:-

- P2 is actually simply copying the file to a remote server for processing
- P2 is supposed to get it there as quickly as possible
- Any mechanism of using API to check lock is only masking the issue since it could get locked in the intervening time [not actually possible in this case - at least in theory]
- I am not sure of the granularity of timestamps in any event (though it shouldn't take much effort to work out)

P2 is fully restartable, as is P1 in fact. My beef really is just that I have to trap the exception - and it's not specific enough to be sure. This does mean that I might - for example - happen to retry when it's a network failure on the copy - and I can't tell from the exception.

Live with it seems to be the right answer.