c#爬虫抓取数据

//发起请求 HttpWebResponse Web_Response = (HttpWebResponse)Web_Request.GetResponse(); //请求完毕以后返回内容 Stream streams = Web_Response.GetResponseStream(); //读取流 StreamReader streamreader = new StreamReader(streams, Encoding.GetEncoding("utf-8")); //读取成string格式内容 string htmls = streamreader.ReadToEnd();

总体来说c# 的爬虫核心在这里 当然如果有反爬的话 就需要加cookie 加proxy 来进行破解

加请求偷这样加 Web_Request.Headers.Add("Connection", "keep-alive"); Web_Request.Headers.Add("Host", "hotels.lvmama.com");

加proxy的话 这样加 WebProxy proxy = new WebProxy("111.13.7.120", 80); Web_Request.Proxy = proxy;

加cookie这样加 Web_Request.CookieContainer = cookie;