最近很热门的话题:你买新能源了吗?连一些大户基民都要买新能源,大概是因为市场目前主线只剩新能源,不买新能源,近期几乎是没什么赚钱效应的。关于基金研究这方面我不太懂行,不过理财都是一样的,收益高,风险也高。收益低的,风险也低。
但是我作为一个爬虫研究者,我可以利用我的技能获取理财数据进行分析,可以帮助大家了解下呀本文就是利用python对某一财经网站的基金进行了爬取,访问网站我们要记住一点就是采集的数据是公开的哦。所以我们今天就是访问的天天基金里的公开数据,本文数据来源:fund.eastmoney.com/
接下来是代码部分,我们主要讲解下如何在爬虫程序里面添加代理,代码来源www.16yun.cn/help/ss_dem…
import org.apache.commons.httpclient.Credentials;
import org.apache.commons.httpclient.HostConfiguration;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.HttpMethod;
import org.apache.commons.httpclient.HttpStatus;
import org.apache.commons.httpclient.UsernamePasswordCredentials;
import org.apache.commons.httpclient.auth.AuthScope;
import org.apache.commons.httpclient.methods.GetMethod;
import java.io.IOException;
public class Main {
# 代理服务器(产品官网 www.16yun.cn)
private static final String PROXY_HOST = "t.16yun.cn";
private static final int PROXY_PORT = 31111;
public static void main(String[] args) {
HttpClient client = new HttpClient();
HttpMethod method = new GetMethod("https://httpbin.org/ip");
HostConfiguration config = client.getHostConfiguration();
config.setProxy(PROXY_HOST, PROXY_PORT);
client.getParams().setAuthenticationPreemptive(true);
String username = "16ABCCKJ";
String password = "712323";
Credentials credentials = new UsernamePasswordCredentials(username, password);
AuthScope authScope = new AuthScope(PROXY_HOST, PROXY_PORT);
client.getState().setProxyCredentials(authScope, credentials);
try {
client.executeMethod(method);
if (method.getStatusCode() == HttpStatus.SC_OK) {
String response = method.getResponseBodyAsString();
System.out.println("Response = " + response);
}
} catch (IOException e) {
e.printStackTrace();
} finally {
method.releaseConnection();
}
}
}
具体的代理配置方式注释里面已经很详细了,大家有需求的可以学习下,有疑问的可以留言大家一起探讨,最后提醒大家,入市需谨慎,股市有风险。