Hey guys,
I'm trying to enhance my coding skills and I've been working on a couple of scraping apps. Thus far I've had no problems logging into websites, but VBulletin is driving me nuts. I've checked the cookies via Wireshark and in VisualStudio and they seem A-OK, but I never actually log in. Edit: I just wanted to add that I've checked the parameters I'm passing and they're correct. I've checked the hash of my password against VBulletin's hash and that's okay.
<-- confused
What stupid stuff am I doing?
Here's my login function:
Which I use to get the logged in cookie so that I can grab html via:
I'm trying to enhance my coding skills and I've been working on a couple of scraping apps. Thus far I've had no problems logging into websites, but VBulletin is driving me nuts. I've checked the cookies via Wireshark and in VisualStudio and they seem A-OK, but I never actually log in. Edit: I just wanted to add that I've checked the parameters I'm passing and they're correct. I've checked the hash of my password against VBulletin's hash and that's okay.
<-- confused
What stupid stuff am I doing?
Here's my login function:
Code:
public static String Login(String username, String password, String formUrl)
{
String formParameters = String.Format("vb_login_username={1}&vb_login_password=&s=&securitytoken=guest&do=login&vb_login_md5password={0}&vb_login_md5password_utf={0}", Md5Hash(password), username);
String cookieheader;
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(formUrl);
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
req.ContentLength = formParameters.Length;
req.CookieContainer = _cookies;
byte[] bytes = Encoding.ASCII.GetBytes(formParameters);
using (Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
WebResponse resp = req.GetResponse();
cookieheader = resp.Headers["Set-Cookie"];
return cookieheader;
}
Which I use to get the logged in cookie so that I can grab html via:
Code:
public static String GetHtmlSource(String url, String cookie)
{
HttpWebRequest request = (HttpWebRequest) WebRequest.Create(new Uri(url));
request.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3";
// request.Headers.Add("Cookie", cookie);
request.CookieContainer = _cookies;
HttpWebResponse response = null;
try
{
response = (HttpWebResponse)request.GetResponse();
String s = new StreamReader(response.GetResponseStream(), Encoding.ASCII).ReadToEnd();
response.Close();
s = s.Replace(Environment.NewLine, "");
return RemoveTabsAndNewlines(s);
}
catch (Exception)
{
if(response != null) response.Close();
return String.Empty;
}
}