OkHttpUrlConnection库编写代码示例

147 阅读1分钟

huake_00219_.jpg OkHttpUrlConnection库编写的爬虫程序,该程序使用Kotlin编写的。

import java.net.HttpURLConnection
import java.net.URL
import java.net.URLConnection
import java.io.BufferedReader
import java.io.InputStreamReader

fun main() {
    val url = URL("https://www.dingtalk.com/")
    val proxy = "http://www.duoip.cn:8000"

    val connection = URLConnection(url)
    connection.setRequestProperty("Proxy-Host", proxy)
    connection.setRequestProperty("Proxy-Port", "8000")

    connection.connect()

    val bufferedReader = BufferedReader(InputStreamReader(connection.getInputStream()))
    var line = bufferedReader.readLine() as String
    while (line != null) {
        if (line.contains("img alt")) {
            val imageUrl = line.substring(line.indexOf("src") + 4, line.lastIndexOf("alt") - 1)
            downloadImage(imageUrl)
        }
        line = bufferedReader.readLine()
    }

    bufferedReader.close()
    connection.disconnect()
}

fun downloadImage(imageUrl: String) {
    val url = URL(imageUrl)
    val connection = URLConnection(url)
    connection.connect()

   
}

代码解释:

  1. 创建URL对象,用于指定要爬取的网站的URL。
  2. 创建一个连接,使用主机和端口。
  3. 连接到目标网站。
  4. 创建一个BufferedReader对象,用于读取目标网站的HTML内容。