C# Get xml site over SSL/TLS - https:// - 【StackMirror】|c#|xml|https|rss|ssl-certificate

I need to be able to read xml/rss from a https web site in a console program.

until now my program supports plain http, and i've been searching around but i cant seem to find an easy way to implement support for https. It would not matter if the site has a valid certificate or not, but i would appriciate to get hints towards how i would check these certificates to.

I might not know too much about this so any hints are appriciated!

what i currently do for http is:

XmlTextReader rssReader;
XmlDocument rssDoc;
rssReader = new XmlTextReader(url);
rssDoc = new XmlDocument();
rssDoc.Load(rssReader);

When trying this on a site without a trusted certificate i get an error stating: "The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel."

string url = "https://somesite.com";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream resStream = response.GetResponseStream();

My program needs to support both trusted and untrusted https sites.

Program is running on a server, and hence has to handle the untrusted https sites in code.

2012-04-05 21:19
by Bjørn

And what happens if you add "s" to "http" in the url? Simply changing it from "http" to "https"? If the site has a trusted certificate, it should just work. If NOT, you'll need to troubleshoot any errors you get - David 2012-04-05 21:24

The site i am testing with does not have a trusted certificate. And my code needs to support both.... - Bjørn 2012-04-05 21:25

which error are you getting or what is the behavior? if you need to 'deal with certificate' error that's one/different issu - NSGaga 2012-04-05 21:25

If the certificate isn't trusted, then the way around it is to explicitly trust that certificate on the machine that the console app runs on. To do that, open up the URL in Internet Explorer, install it from there. There are instructions here: http://www.m86security.com/KB/KnowledgebaseArticle13446.aspx (steps may look different depending on the version of I.E. - David 2012-04-05 21:27

For the certificate issue try the following...

ServicePointManager.ServerCertificateValidationCallback += new System.Net.Security.RemoteCertificateValidationCallback((s, ce, ch, ssl) => true);

...somewhere at the start - or before doing the request.

That's basically validating any certificate unconditionally, a bit simplified.

EDIT: that's 'to blindly' trust (and is of global character for your app) - proper implementation would handle the parameters - or entails implementing ICertificatePolicy to specifically deal with different hosts/certificates.

EDIT (certificates): as to how the certificates and SSL actually work - and related to the above (based on the comments/discussion)...

http://www.verisign.com/ssl/ssl-information-center/how-ssl-security-works/index.html
How does SSL really work?
https://superuser.com/questions/84572/public-key-encryption

2012-04-05 21:30
by NSGaga

This can be just: ServicePointManager.ServerCertificateValidationCallback = (s, ce, ch, ssl) => true;porges 2012-04-05 21:31

@Porges :) of course - it's a matter of style, that's an old piece of code from before, when we used to specify the delegate etc.. But agre - NSGaga 2012-04-05 21:32

Wow! It was just that simple! Thank you! :)

Would you mind explaining what this does? To me it looks like an event handler - Bjørn 2012-04-05 21:35

Yes simply it handles the certificate validation in a custom way - 'An application can set the ServerCertificateValidationCallback property to a method to use for custom validation by the client of the server certificate' from 'http://msdn.microsoft.com/en-us/library/system.net.servicepointmanager.servercertificatevalidationcallback.aspx'. Where there are problems it simply fails (and problem could be due to expired/invalid certificate etc.) and you need to handle that, something along those lines - NSGaga 2012-04-05 21:51

Sort of defeats at least part of the purpose of a certificate doesn't it? The cert not only provides encryption (https) but it also helps to prevent main-in-the middle attacks, where the malicious website intercepts the requests and alters data. Blindly trusting all certs takes away that portion of the cert's purpose, doesn't it? Wouldn't it make more sense to specifically install and therefore trust thos one certificate - David 2012-04-05 21:55

defeats the entire purpose :) - sure - but this is a quick fix, and I'm not sure you can always trust the certificate properly. I often stumble on this and it's always for sites you know you're dealing with, API server etc.. And if the certificate expires or is invalid - you can't trust it really I think - NSGaga 2012-04-05 21:59

What would really be the difference between ignoring certificates that fails, or installing them, and then trusting it? Will this "quick fix" not use encryption then? I have always believed that when you run over https it always uses encryption? It would just not be validated from a "trusted" source - Bjørn 2012-04-05 22:09

I improved/added that to the answer. Certificate ensures that the site "you're talking to" is who it says it is - SSL both encrypts and verifies your communication - i.e. ensures that a) no-one (theoretically) can read it or b) protects you from others 'impersonating' - e.g. if someone intercepts your traffic and injects a false or malicious content, it can still be encrypted, if you trust it unconditionally - NSGaga 2012-04-05 22:18

And when running ServicePointManager.Server...... it encrypts, but it blindly trusts that it's the source i want to talk to, that i really talk to. or does the encryption not happen when running the command? (I am sorry for asking over and over again, i just want to be perfeclty clear) So the encryption key (certificate) would be not secure, but the connection is still encrypted, is what i am asking i guess... - Bjørn 2012-04-05 23:10

I just updated the answer - it's not that simple matter, but better read from those expert on that - or maybe spawn another question with specifics, cheers - NSGaga 2012-04-06 16:19

You'll have to send an HttpWebRequest or use HttpClient. Both of which are designed for making/negotiating these connection.

Possible Dupe: How to load xml from https using XmlTextReader

How do I use WebRequest to access an SSL encrypted site using https?

HttpWebRequest with https in C#

2012-04-05 21:25
by Rob Rodi