添加依赖
<dependency>
<groupId>commons-httpclient</groupId>
<artifactId>commons-httpclient</artifactId>
<version>3.1</version>
</dependency>
基于 Flink 服务提交任务并执行时需要的依赖包
基于 flink 服务器提交任务前,先上传依赖包到 flink 的 lib 目录下;然后重启 flink 服务,使 jar 进行加载;否则会出现 ==ClassNoFoundException== 的异常。
- commons-codec-1.2.jar
- commons-httpclient-3.1.jar
- commons-logging-1.0.4.jar
构建RestfullSink参数实例
public class RestfulSink implements Serializable {
private static final long serialVersionUID = 41026197876622956L;
private String url;
public String getUrl() {
return url;
}
public RestfulSink(Object obj) {
final JSONObject json = JSONObject.parseObject(obj.toString());
this.url = json.getString("url");
}
}
构建自定义RestfulMessageSink
基于继承 RichSinkFunction< T > 抽象类实现自定义Sink,实现方法有三个:
- open():构建sink节点时最先执行的方法,用于实现一些初始化动作。
- invoke():执行节点时执行,用于实现具体业务逻辑。
- close():关闭节点回收资源时执行,用于资源的回收。
import java.io.IOException;
import java.util.Iterator;
import java.util.Map;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.methods.PostMethod;
import org.apache.flink.configuration.Configuration;
import org.apache.flink.streaming.api.functions.sink.RichSinkFunction;
import com.ygsoft.dataprocess.exception.RestfullExcepiton;
import com.ygsoft.dataprocess.vo.sink.RestfullSink;
public class RestfulMessageSink extends RichSinkFunction<Map<String, String>> {
private static final long serialVersionUID = -905405395688757837L;
private RestfulSink restfulSink;
private HttpClient httpClient;
public RestfulMessageSink(final RestfulSink restfulSink) {
this.restfulSink = restfulSink;
}
@Override
public void open(final Configuration parameters) throws Exception {
super.open(parameters);
httpClient = new HttpClient();
}
@Override
public void invoke(Map<String, String> value, Context context) throws Exception {
PostMethod postMethod = new PostMethod(restfullSink.getUrl());
postMethod.getParams().setContentCharset("UTF-8");
Iterator<Map.Entry<String, String>> iterator = value.entrySet().iterator();
while (iterator.hasNext()) {
Map.Entry<String, String> next = iterator.next();
postMethod.addParameter(next.getKey().equals("value")? "data" : next.getKey(), next.getValue().toString());
}
try {
httpClient.executeMethod(postMethod);
/*int code = httpClient.executeMethod(postMethod);
if(code != 200) {
System.out.println(postMethod.getResponseBodyAsString());
}*/
} catch(IOException e) {
throw new RestfullExcepiton("restful接口调用失败");
}
}
@Override
public void close() throws Exception {
super.close();
}
}