Alright, let's consider this resolved.

Yes, the bitbucket thing is a known issue (took me a bit to realise they were at
fault and not my code). I'm working with them to help them figure out where in
their authentication implementation they are going wrong.

mpm: you're right -- total pilot error on my part.  Sorry!

Once I fixed my error, I can push and pull to https through my proxy.  Excellent
-- thanks folks!

No, it means you're probably getting an HTML error message from your server.

Ah, ok -- scrap bb.org, then.

I tried a different hg / https:

$ /tmp/bogus/bin/hg --debug fetch osl
using https://www.open-mpi.org/hg/auth/hgwebdir.cgi/jsquyres/opal-sos/
proxying through http://proxy-sjc-2.cisco.com:80
http auth: user jsquyres, password *********
sending between command
http auth: user jsquyres, password *********
requested URL:
'https://www.open-mpi.org/hg/auth/hgwebdir.cgi/jsquyres/opal-sos/?pairs=0000000000000000000000000000000000000000-0000000000000000000000000000000000000000&cmd=between'
(falling back to static-http)
proxying through http://proxy-sjc-2.cisco.com:80
http auth: user jsquyres, password *********
http auth: user jsquyres, password *********
abort: requirement '<?xml version="1.0" encoding="ascii"?>' not supported!
$

Is this telling me that I need a more recent hg on the server side?  I believe
it's currently hg 1.1.1 on the server.  (forgive the n00b question...)

I believe the bitbucket problem is is a known problem on bitbucket's end.

As soon as I submitted the last message, I remember about the --debug switch --
duh.  I also forgot to mention that /tmp/bogus/bin/hg is my install of crew. 
Sorry for the noise, but here's "push" output with --debug enabled:

$ /tmp/bogus/bin/hg --debug push https://jsquyres@bitbucket.org/jsquyres/proxy-test/
using https://bitbucket.org/jsquyres/proxy-test/
proxying through http://proxy-sjc-2.cisco.com:80
http auth: user jsquyres, password not set
sending between command
pushing to https://jsquyres@bitbucket.org/jsquyres/proxy-test/
sending capabilities command
capabilities: unbundle=HG10GZ,HG10BZ,HG10UN lookup changegroupsubset
sending heads command
searching for changes
common changesets up to ca9e055628d5
1 changesets found
list of changesets:
abdc4e1f661b761cf807a52bcba4e4b1318798de
sending unbundle command
sending 314 bytes
http authorization required
realm: Bitbucket.org HTTP
user: jsquyres
password: <MYPW>
http auth: user jsquyres, password *********
http auth: user jsquyres, password *********
http auth: user jsquyres, password *********
http auth: user jsquyres, password *********
http auth: user jsquyres, password *********
http auth: user jsquyres, password *********
abort: authorization failed
$

Output is pretty much the same with <MYPW> in the URL:

$ /tmp/bogus/bin/hg --debug push
https://jsquyres:<MYPW>@bitbucket.org/jsquyres/proxy-test/
using https://bitbucket.org/jsquyres/proxy-test/
proxying through http://proxy-sjc-2.cisco.com:80
http auth: user jsquyres, password *********
sending between command
pushing to https://jsquyres:***@bitbucket.org/jsquyres/proxy-test/
sending capabilities command
capabilities: unbundle=HG10GZ,HG10BZ,HG10UN lookup changegroupsubset
sending heads command
searching for changes
common changesets up to ca9e055628d5
1 changesets found
list of changesets:
abdc4e1f661b761cf807a52bcba4e4b1318798de
sending unbundle command
sending 314 bytes
http auth: user jsquyres, password *********
http auth: user jsquyres, password *********
http auth: user jsquyres, password *********
http auth: user jsquyres, password *********
http auth: user jsquyres, password *********
http auth: user jsquyres, password *********
abort: authorization failed
$

I installed a fresh python 2.6.2 on RHEL4U4 (unfortunately, some of us are stuck
back on RHEL4 with ancient Python 2.3, so I had to install my own Python 2.6.2
by hand) and pulled down crew as of 8638:c6483eec6092 (I can see the noted
commit -- 8590:59acb9c7d90f -- in hg log).

I can "hg clone" and "hg pull" from https through my proxy now.  Woo hoo!

However, "hg push" doesn't seems to work -- I cannot authenticate properly.  I
have tried both:

$ /tmp/bogus/bin/hg push https://jsquyres:<MYPW>@bitbucket.org/jsquyres/proxy-test/
pushing to https://jsquyres:***@bitbucket.org/jsquyres/proxy-test/
searching for changes
abort: authorization failed

and

$ /tmp/bogus/bin/hg push https://jsquyres@bitbucket.org/jsquyres/proxy-test/
pushing to https://jsquyres@bitbucket.org/jsquyres/proxy-test/
searching for changes
http authorization required
realm: Bitbucket.org HTTP
user: jsquyres
password: <MYPW>
abort: authorization failed

So it seems like the underlying problem is partially worked around, but not
entirely...?

Is there any further info that I could provide to help?

A fix for this should be in crew as 59acb9c7d90f. I have made extensive tests
with squid3, but please test whether it works for you, in particular if you're
using a different proxy server. 

Please note that crew has dropped support for 2.3, so people there would have to
make a custom version based off of 1.2.1 with 59acb9c7d90f transplanted (or
equivalent) - there are no current plans to create a 1.2.2 release, according to
mpm.

FWIW, it would be great if you could work around it in Mercurial for those of us
who are stuck using very old versions of Python (e.g., 2.3.x in RHEL4).

Well we know what the issue is: Python's urllib2 is busted. So someone needs to
either fix it in Python or work around it in hg. It might be possible to do the
latter with something like

http://code.activestate.com/recipes/456195/

but I'm not in a position to test it.

Sadly, it does not.  :-(

Here's a bunch of info showing that it doesn't work on a Mercurial 1.2.1 with
the patch applied:

[11:08] svbu-mpi:~/hg % hg --debug clone
https://jsquyres@bitbucket.org/jsquyres/proxy-test
using https://bitbucket.org/jsquyres/proxy-test
proxying through http://proxy-sjc-2.cisco.com:80
http auth: user jsquyres, password not set
sending between command
abort: HTTP Error 400: Bad Request

Let's verify that the patch is applied:

[11:08] svbu-mpi:~/hg % where hg                                               
/opt/mercurial/1.2.1/bin/hg
[11:08] svbu-mpi:~/hg % grep _tunnel_host
/opt/mercurial/1.2.1/lib64/python2.3/site-packages/mercurial/keepalive.py
                if hasattr(req, '_tunnel_host') and req._tunnel_host:
                    h.set_tunnel(req._tunnel_host)
[11:08] svbu-mpi:~/hg % ls -l
/opt/mercurial/1.2.1/lib64/python2.3/site-packages/mercurial/keepalive.py*
-rw-r--r--  1 root root 22109 May  5 11:03
/opt/mercurial/1.2.1/lib64/python2.3/site-packages/mercurial/keepalive.py
-rw-r--r--  1 root root 24144 May  5 11:03
/opt/mercurial/1.2.1/lib64/python2.3/site-packages/mercurial/keepalive.pyc

Here's my Python version (RHEL4U4):

[11:08] svbu-mpi:~/hg % python -V
Python 2.3.4
[11:09] svbu-mpi:~/hg % 

Is there anything else I can send to help diagnose?

Can you try the patch in msg7779 and let us know if it works for you?

Greetings all -- is there any progress on this issue?  I'm running into this
exact problem and was just wondering if there was any progress since Dec 2008.

Thanks!

The patch seems correct to me. However, things are evolving a bit on the Python
side (http://bugs.python.org/issue1424152) and a close synchronization of this
issue and the Python one are important.

So this should do the job?

diff --git a/mercurial/keepalive.py b/mercurial/keepalive.py
--- a/mercurial/keepalive.py
+++ b/mercurial/keepalive.py
@@ -237,6 +237,8 @@
             else:
                 # no (working) free connections were found.  Create a new one.
                 h = http_class(host)
+                if hasattr(req, '_tunnel_host') and req._tunnel_host:
+                    h.set_tunnel(req._tunnel_host)
                 if DEBUG: DEBUG.info("creating new connection to %s (%d)",
                                      host, id(h))
                 self._cm.add(host, h, 0)

It's harmful when python patch isn't applied. But testing for 
_tunnel_host attribute presence should be sufficient to support both 
cases.

It's harmful when python patch isn't applied. But testing for _tunnel_host
attribute presence should be sufficient to support both cases.

@phil
Should the patch be applied to the keepalive.py version used in mercurial? Or is
it harmful when the python patch isn't applied?

Recipe to have it work:

As mentionned earlier, urllib2/httplib do not support it, and a patch is
provided here:
http://bugs.python.org/issue1424152
Apply it.

but mercurial overloads some of the logic, and a part of the patch must be
reported there, so apply this one too:
# HG changeset patch
# User Phil <phil@secdev.org>
# Date 1214998663 -7200
# Node ID 06270abc1d2dbe81c9caa590779209f5d3ba808d
# Parent  e81d2bd669088aa22e3437c610d216500ffeb02c
Code to support https connections through proxy

urllib2/httplib does not support those connections. A patch
is proposed here:
http://bugs.python.org/issue1424152

Because Mercurial overloads some parts of urllib2, a part
of the patch must be transfered to the overloading mercurial code.

diff --git a/mercurial/keepalive.py b/mercurial/keepalive.py
--- a/mercurial/keepalive.py
+++ b/mercurial/keepalive.py
@@ -237,6 +237,8 @@
             else:
                 # no (working) free connections were found.  Create a new one.
                 h = http_class(host)
+                if req._tunnel_host:
+                    h.set_tunnel(req._tunnel_host)
                 if DEBUG: DEBUG.info("creating new connection to %s (%d)",
                                      host, id(h))
                 self._cm.add(host, h, 0)

This was first brought up on the mercurial mailing list and consensus seems to 
be this is really a bug in the Python urllib/urllib2, see

http://bugs.python.org/issue1424152

Just as a clarification, the combination of mercurial, https and a proxy does 
work for plenty of people. In my case it fails because the proxy at my office is  
very strict.

Adding nosy from issue606.

mpm proposed using something like this:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/456195

See also issue606.

I can confirm with crew-stable and crew with python 2.4 on Debian etch.

hg sends a request GET https://... instead of using CONNECT.

I'm trying to push to a repo pusblished over https through squid. The push fails:

--
moya@gloria:~/src/foo-trunk$ hg --verbose --debug push https://hg.foo.org/foo-trunk
using https://hg.foo.org/foo-trunk
proxying through http://localhost:3128
pushing to https://hg.foo.org/foo-trunk
sending capabilities command
abort: HTTP Error 501: Not Implemented
--

The same push works if I do it not behind the proxy. The squid log have the
following

1202287933.801      3 127.0.0.1 TCP_DENIED/501 1501 GET
https://hg.foo.org/foo-trunk?cmd=capabilities - NONE/- text/html

Searching the web for 'GET https' I found this
http://wiki.squid-cache.org/SquidFaq/TroubleShooting#head-03d5d82b9c2b2e558084f4ba72b226a711639d62

I'm using mercurial 0.9.5.

Regards,
maykel