Windows Vista VPN: A Step Backward

by alec on October 13, 2006

One of the most frustrating aspects of Windows XP is how difficult it can be to get a VPN running.  Unless you are running a full set of domain services,  the process is a little bit like divining the future amongst chicken entrails: messy and unpleasant with a heavy dose of guesswork.  It typically involves manual manipulation of firewall ports, manual mapping of hosts on the VPN side, and a lot of shrewd guessing.

Having said that, for some time I’ve been successfully running Windows XP, and the Windows OneCare Live security package, which does firewall, antivirus, and spyware protection, as well as nagging me about backups.  It’s a nice integrated tool.  In order to get it to work with our VPN, I needed to open the GRE protocol port — helpfully renamed Microsoft VPN in later builds.  It was a fair work of divination to make that happen because:

1) The Windows XP VPN client doesn’t actually provide any useful information when it’s blocked.  In this particular case, the VPN connects, informs you that it’s verifying your password, and fails on password verification.  There are obviously many possibilities at this point, including the fact that you simply might have mistyped the password.  The “oh-so-informative” error message 619 provides the following possible clues:

There are several possible reasons why a connection to the remote computer could not be established:

  • The remote computer might have been too busy. Wait a few minutes and try the connection again.
  • If you are trying to establish a dial-up connection, you might have tried to redial before the modem fully disconnected. Wait a short time and try your call again.
  • If you are trying to establish a connection by using a modem, the modem might not be functioning properly. For more information, see Troubleshooting modems.
  • If you are using a device such as a router, a hub, or a network adapter for network address translation (NAT), the device might not be functioning properly. If the device provides firewall capabilities, the device might be blocking the connection. Consult the documentation for the device.

2) The Windows Live OneCare firewall doesn’t inform you which port it has blocked.  It simply blocks.

That was several months ago. I’ve been successfully running Windows Vista Beta 2, and then RC 1 with the TrendMicro PC-Cillin beta.  Until recently,  PC-Cillin was the only solution for Windows Vista. However, a couple of days ago, I upgraded the PC to Windows Vista RC2, and the just-released Windows OneCare Live 1.5 beta.  That’s when the nightmares started.

You see, unlike the mostly unhelpful messages provided by Windows XP, Windows Vista provides you with no information.  It says “Failed to connect”, and then offers “Diagnose the problem”, which unhelpfully told me that it couldn’t find anything wrong.

After several attempts to get the correct ports open in Windows OneCare Live, I gave up.  Turning the firewall off helped me to determine that the problem was indeed the firewall, and I have now reverted to PC-Cillin.

There’s a bug in Windows OneCare Live’s firewall support.  More importantly, though, it’s nearly impossible to diagnose in Windows Vista. That’s a huge usability problem.

{ 24 comments… read them below or add one }

Brandon Kelly October 24, 2006 at 6:06 am

Agreed. I am running Vista RC1 with Windows OneCare Live 1.5 and am having the exact same problem. I also went through the steps to open up the needed port with OneCare on Windows XP with some frustration (and success) and find the new "Diagnose Problem" feature to be utterly useless.

It seems Microsoft is going the way of Apple — very pretty O/S with loads of features and little to no feedback. Just a warm, friendly message telling you "no" in the politest way possible. This is undoubtedly the result of the marketing department gone wild.

Reply

Adrian Cucu October 24, 2006 at 1:37 pm

Vista RC2, Live OneCare 1.5 beta
[How to]
1. Display OneCare's settings box, and select the 'Firewall' tab.
2. Choose 'Firewall connection tool' and then check the item 'Microsoft VPN – use a Microsoft virtual private network'.
3. Choose ‘Advanced settings…’ and define the following three 'Ports and protocols' rules:
a) Protocol: TCP, Port Range: 1723-1723, Scope: Internet
b) Protocol: UDP, Port Range: 500-500, Scope: Internet
c) Protocol: UDP, Port Range: 4500-4500

Reply

Alec October 24, 2006 at 5:18 pm

Thanks Adrian. I am still mystified as to how I am supposed to figure this out, but glad to have the information.

Reply

Sean December 3, 2006 at 1:00 pm

Alec,

How did you configure PC-Cillin to work with the Microsoft VPN. I have been playing around with Pc-Cillin and have been unsuccessful in configuring it to allow a Windows XP VPN connection.

Reply

Alec December 3, 2006 at 2:01 pm

Sean, I just installed it. Worked like a charm. Sorry, I know that's not belpful.

Reply

AlliXSenoS December 18, 2006 at 7:05 am

you can fix the problem (I have) by turning on Windows Live OneCare / Change Settings / Firewall / Firewall connection tool / VPN – connect to another computer over a virtual private network.

Reply

Alberto Di Meglio January 10, 2007 at 10:26 am

Alec,

What VPN method are you using, PPTP or IPSec? The issue with PC-Cillin and PPTP VPN is still there for me in the latest beta release of PC-Cillin 2007 on Windows Vista. According to PC-Cillin support this is a known issue (the PC-Cillin firewall doesn't manage the GRE 47 protocol, which means that the initial negotiation fails even if you explicitly open the port 1723)

Reply

Richard Ã&hel February 19, 2007 at 4:37 am

Hi.

I had the exact same problem running Vista (business) and Windows One Care 1.5.
I did, however, stumble upon a sollution.
Here's what i did:
1. Open Windows Live OneCare.
2. Click: "Change OneCare Settings" –> Firewall
3. Click the button: "Firewall connection tool".
4. Make shure the checkbox "VPN – connect to another computer over a virtual private network" is NOT empty.
Just click ok all the way out, and your VPN connection is now working.

Reply

Tony March 18, 2007 at 1:46 pm

I agree, there are many hassles setting up a VPN service on Windows XP and Vista. Users should consider other VPN clients, like Hamachi to help calm their nightmares about VPN service.

Nationwide VPN

Reply

olrac May 25, 2007 at 1:21 pm

I am having the same issue with the vpn as well it works fine with xp and not with Vista Ultimate. Evertime I connect, I get the same message :

Error 732. Your computer the the remote computer could not agree on PPP control protocols.

I have not found any solutions, if anyone has any ideas they would be greatly appreciated.
thanks

Reply

Sean June 8, 2007 at 2:07 pm

using windows vista premium to try and set up a VPN every time I do so it will say internet explorr is not connected. If I manually type in the server address it allows me in but then I can reply and it claims site has a certificate error (it doenst as I operate vpn) on xp rom the same server ! Ideas?

Reply

Tim June 29, 2007 at 3:16 pm

I cannot get a VPN connection at all usiing XP Pro as client and Vista Business as server. all firewalls off on both ends (for now), and all necessary ports open. using a linksys wrt150n…vPN passthru enables, and ports open. i am absolutely clueless…since it works fine between (2) xP pro setups.

triied to PINg the server, nothing. no ping as if it were dead…but i can remote desktop no problem to the server? so clueless yet again. i guess vista business is broke with VPN or soemthing? or something is blocking it even though firewalls off and ports open???

Reply

Alec June 29, 2007 at 6:15 pm

Tim – random thought — are both PC's on the same domain or workgroup name? They've changed the rules for Vista Business edition so that if the workgroup name doesn't match, servers are invisible to each other.

Reply

Tim July 2, 2007 at 7:53 pm

yes, that was the first thing I checked. thanks!

tried calling microshaft today to use the first of the 90 days support…well…turns out only for easy basic type help…vpn stuff they want $250?! jeez. so going to keep digging around the net until i find a solutioin…funny thing…works fine between two xp pro machines…but between a xp pro and a vista business (server/host)…NO GO! there has to be somthing in vista preventing it…just can't find out what?

Reply

Cliff October 12, 2007 at 4:20 am

It seems that Windows Vista uses v2 of CHAP and this causes problems when trying to connect via VPN, especially to older VPN solutions.

Reply

john October 21, 2007 at 8:29 pm

There is way too many problems with vista, just in general. Thats why i bought a Mac. I just decided to switch to another high speed internet service provider as well. They are cheap and reliable. Check them out: http://ispsurvey.com

Reply

George December 30, 2007 at 5:35 pm

I have Live onecare 2.0 and blocks all my gaming ports. I used to have norton antivirus and it will automatically adjust the ports for the game. Now I need to configure all the ports which is a pain and I don't even know which ones are they. Any help?

Reply

Jason March 16, 2008 at 6:29 am

I've been trying to setup VPN using a Vista Ultimate computer as the server and have a Vista Pro computer trying to connect to through VPN with no success as well. I have opened up TCP ports, should I be opening up UDP ports as well? This is giving me quite a headache..lol. I have also tried to connect using a few different XP home and pro versions…

Reply

vpn service May 27, 2008 at 4:08 am

Vista is a really fed up me. It's a really sucks OS. I'am downgrading to XP on my new notebook.

Reply

Daniel Burnsn August 25, 2008 at 9:40 pm

Sean and anyone else who might still be wondering, in answer to your XP PPTP VPN questioon and configuring PC Cillin. I have just got this working. To do so, go into the firewall settings and into whichever profile pc cillin is using, eg 'Direct Internet Connection' and click Edit, Under network control tab, click Add, Call it something sensible like VPN PPTP, set connection outgoing, action allow, type TCP, Specified port 1723, and click ok. Then from network control tab again, click Add, Call it GRE Protocol, make connection outgoing, action allow, ***set protocol to 'custom' then enter protocol number 47, then click ok. Then try again. Sorry, instructions aren't pretty but hopefully they work, don't have time for anything nicer.

cheers,
Daniel

Reply

Angela March 10, 2010 at 6:16 am

Microsoft is awesome. They are great at everthing they make.

Reply

MaxRegistryCleaner July 1, 2010 at 2:40 pm

I have a pptp-vpn on my m0n0wall gateway. When I used Windows XP on my laptop it worked flawless to connect to the VPN wherever I was. Now I’m on Vista and when I connect to my VPN I get a dedicated ip (v4) from the VPN-server, but then my local network connection that connects me to the internet dies somehow…and that makes the vpn connection die too…I have no idea why it behaves like this and it drives me nuts :(

Any suggestions?

Reply

Super VPN Service July 19, 2010 at 4:36 pm

I think all problems with vista are solved with win 7.VPN connection works perfect with that OS.

Reply

Alex January 11, 2012 at 12:21 pm

There are some issues with VPN in VIsta. A while ago I solved it by installing SP1 for Vista. Anyway, I switched to W7 now and a new VPN provider, http://www.sunvpn.com/, works much better now…

Reply

Leave a Comment

Previous post:

Next post:

Alec on LinkedIn Alec on Twitter Alec on Facebook Calliflower on Youtube RSS Feed Contact me