Issue1820

Title Avira Antivirus causing ERROR_USER_MAPPED_FILE on Windows
Priority wish Status chatting
Superseder Nosy List GrayNM, Jean-Louis, abuehl, mpm, pmezard, sborho
Assigned To Topics windows

Created on 2009-09-03.09:08:24 by GrayNM, last changed 2010-01-14.10:11:27 by abuehl.

Messages
msg11463 (view) Author: abuehl Date: 2010-01-14.10:11:27
See also http://selenic.com/pipermail/mercurial/2010-January/029680.html
msg11462 (view) Author: abuehl Date: 2010-01-14.09:46:28
@Jean-Louis: excluding all repository root directories seems sensible. 
Probably easiest to have them all in a common directory and then exclude 
that. You might excluding that from the indexing Service as well.
msg11458 (view) Author: Jean-Louis Date: 2010-01-13.17:27:36
> Yes, Avira Antivir Personal.
> I suspect that this is antivirus influence, but anyway disabling it 
> for any repository change is not good solution.

Hi, sorry for my bad english.

There are settings in Avira for exclude directory scanning

Configuration->expert mode->guard->scan->exceptions

is a good workaround if I add my repository directory?

I'm a linux user, but for some days I have to use windows... but i would
continue to use mercurial.
msg10918 (view) Author: GrayNM Date: 2009-11-08.08:58:44
Sorry for absence, I had absolutely no time last month.

1. In general multitask OS guaranteer that single file operation is atomic
not sequence of them. So claims that OS must provide this contract are
incorrect. Want we or not but file system is a shared resource and it is
necessary to process correctly file access errors due to locks.

2. I wrote my point in my very first post. AV scanners behavior is
predictable: they locks a file for a short period of time and then release
it. So several retries is enough to understand: is this FS failure and the
file is lost or is this only temporary lock by AV and processing can continue.
msg10592 (view) Author: abuehl Date: 2009-09-28.22:49:03
See also Matt's posting
http://selenic.com/pipermail/mercurial-devel/2009-September/015644.html
msg10565 (view) Author: abuehl Date: 2009-09-23.22:18:45
Now we also have this in Issue1840
msg10553 (view) Author: abuehl Date: 2009-09-23.15:51:27
See also potentially similar http://bitbucket.org/tortoisehg/stable/issue/580/

But there, it was AVG free (http://free.avg.com/) and the error message was
different.
msg10470 (view) Author: abuehl Date: 2009-09-04.16:29:52
lowering prio to wish, since this is not a bug of mercurial
msg10469 (view) Author: GrayNM Date: 2009-09-04.16:10:05
1. Scan-on-write is a reasonable scanner policy. This is not unique feature
of Avira. Many other scanner have the same one. You want to wait for another
bugs of this sort with another AV? Want we or not but antivirus scanner can
delay or interrupt any file operation at any time, at least on Windows.

Apropos, Avira manual clearly stated that with this option on files after
write will be unavailable for a short period of time.

2. File is not locked forever. It's locked only for a short period of time.

3. I'm using Avira and mercurial together for about an year. This is first
time I'd faced this bug. Of course this is the first time I'm applying patch
with 6000 files. But several times there were patches with almost 3000 files
and all was ok. So absence of issues with another AV doesn't mean that they
will not appear.

4. I'm surprised why you do endless open-write-close on merge/state instead
of write-flush? May be this will be a little faster at the same time?

5. Correct me if I'm not right but I think that the main sense is to make
mercurial as reliable as possible. This workaround is not large price for AV
problems absence.
msg10468 (view) Author: abuehl Date: 2009-09-04.13:48:29
Adding mpm (Matt Mackall) to nosy.

@GrayNM:

If a broken virus scanning process is holding on files such that regular 
programs can't access them any more, we are lost. Don't use such stupid 
scanners in the first place.

You might want to take this up with the makers of the broken scanner.

Adding workarounds of the sort of "let's see if we are allowed to write to 
this file now and try again later if not" for such broken products to 
mercurial doesn't make any sense.
msg10467 (view) Author: GrayNM Date: 2009-09-04.13:23:56
This is a very strange decision.
This error can be caused by any other antivirus software with aggressive
on-write scanning too.

I propose another solution: open file several more times if got this error.
In that case mercurial will be protected from similar problems in future.
Realization of it is quite simple: I wrote necessary code block in first
message - that's all.
msg10460 (view) Author: abuehl Date: 2009-09-03.11:13:05
Just for reference: TortoiseHG 0.8.1. contains mercurial 1.3.1
msg10459 (view) Author: abuehl Date: 2009-09-03.11:02:45
We've had reports of errors caused by

  avira antiv personal free edition Produktversion 8.1.00.296

in the past already (last time this was on 2008-04-29, reported and
confirmed by a user on IRC channel #mercurial).

I strongly recommend not to use this antivirus product together with
mercurial.

I consider this issue resolved. Don't use avira antiv personal.
msg10458 (view) Author: GrayNM Date: 2009-09-03.09:54:54
Yes, Avira Antivir Personal.
I suspect that this is antivirus influence, but anyway disabling it for any
repository change is not good solution.
msg10455 (view) Author: abuehl Date: 2009-09-03.09:39:33
Do you have any antivirus SW installed and running?
msg10453 (view) Author: GrayNM Date: 2009-09-03.09:08:24
Yesterday I got fail while updating my working dir.
I've downloaded changes from remote repository successfully and tried to
apply them. That was a big update, almost 6000 files.
Error message was "abort: No such file or directory:
E:\Work\Gray\Projects\Global\contrib\.hg\merge/state" but file was present
in file system.
OS: WinXP SP3, TortoiseHG 0.8.1.
Experiment with Fedora 11 shows that linux version works correct.

Trace shows that error occurs in osutil:posixfile() routine.
After some investigation I've found that CreateFile in very rare cases
returns ERROR_USER_MAPPED_FILE. I think that is because system still not
released file when trying to open it again.

I've added ability to retry open operation like this:

another_try:
	handle = CreateFile(name, access,
			    FILE_SHARE_READ | FILE_SHARE_WRITE |
			    FILE_SHARE_DELETE,
			    NULL,
			    creation,
			    FILE_ATTRIBUTE_NORMAL,
			    0);

	if (handle == INVALID_HANDLE_VALUE) {
		ierr=GetLastError();
		if((ierr==ERROR_USER_MAPPED_FILE)&&(retrycnt<5))
		{
			++retrycnt;
			Sleep(200);
			goto another_try;
		}
		PyErr_SetFromWindowsErrWithFilename(ierr, name);
		goto bail;
	}

After that update was ok. 200ms sleep I got just to be sure. I think it can
be much less.
History
Date User Action Args
2010-01-14 10:11:27abuehlsetnosy: mpm, sborho, pmezard, abuehl, GrayNM, Jean-Louis
messages: + msg11463
2010-01-14 09:46:28abuehlsetnosy: mpm, sborho, pmezard, abuehl, GrayNM, Jean-Louis
messages: + msg11462
title: Avira Antivir causing ERROR_USER_MAPPED_FILE on Windows -> Avira Antivirus causing ERROR_USER_MAPPED_FILE on Windows
2010-01-13 17:27:36Jean-Louissetnosy: + Jean-Louis
messages: + msg11458
2009-11-08 08:58:44GrayNMsetnosy: mpm, sborho, pmezard, abuehl, GrayNM
messages: + msg10918
2009-09-28 22:49:03abuehlsetnosy: mpm, sborho, pmezard, abuehl, GrayNM
messages: + msg10592
2009-09-23 22:18:45abuehlsetnosy: + sborho
messages: + msg10565
2009-09-23 15:51:28abuehlsetnosy: mpm, pmezard, abuehl, GrayNM
messages: + msg10553
2009-09-05 11:48:10pmezardsetnosy: + pmezard
2009-09-04 16:29:52abuehlsetpriority: urgent -> wish
nosy: mpm, abuehl, GrayNM
messages: + msg10470
2009-09-04 16:10:06GrayNMsetnosy: mpm, abuehl, GrayNM
messages: + msg10469
2009-09-04 13:48:29abuehlsetnosy: + mpm
messages: + msg10468
2009-09-04 13:23:56GrayNMsetstatus: resolved -> chatting
nosy: abuehl, GrayNM
messages: + msg10467
2009-09-03 11:28:39abuehlsetstatus: chatting -> resolved
nosy: abuehl, GrayNM
2009-09-03 11:13:05abuehlsetstatus: resolved -> chatting
nosy: abuehl, GrayNM
messages: + msg10460
2009-09-03 11:04:36abuehlsetnosy: abuehl, GrayNM
title: Windows misbehavior -> Avira Antivir causing ERROR_USER_MAPPED_FILE on Windows
2009-09-03 11:02:45abuehlsetstatus: chatting -> resolved
nosy: abuehl, GrayNM
messages: + msg10459
2009-09-03 09:54:54GrayNMsetnosy: abuehl, GrayNM
messages: + msg10458
2009-09-03 09:43:47abuehlsettopic: + windows
nosy: abuehl, GrayNM
2009-09-03 09:39:33abuehlsetstatus: unread -> chatting
nosy: abuehl, GrayNM
messages: + msg10455
2009-09-03 09:38:58abuehlsetnosy: + abuehl
2009-09-03 09:08:24GrayNMcreate