一个解决curl命令下载文件为空的小技巧,项目调试中很适用

266 阅读4分钟

引言

最近在进行线上项目调试的过程中需要安装调试工具,因为没有安装wget命令,只安装了curl工具, 所以尝试使用curl -O 命令下载调试工具,结果发现下载的文件为空,通过查看官方手册,发现 curl 默认是不追踪重定向地址的,需要添加 -L 参数才能达到自动跟踪重定向地址的目的,本文 将具体的排查和解决问题的步骤整理成此篇文章分享给大家。

1.通过curl -O 下载远端文件

1.1执行curl -O下载远端文件

预期是能正常将jaatch包下载至当前目录,但执行结果是下载文件为空

curl -O  https://github.com/jattach/jattach/releases/download/v2.1/jattach

download-nil.jpg

1.2 通过curl -v 查看详细的请求和响应的信息

curl -O -v https://github.com/jattach/jattach/releases/download/v2.1/jattach

可以看到远端有一个302重定向的响应,curl没有重新请求这个重定向的地址

curl-302.jpg 详细请求、响应信息如下

  # curl -O -v https://github.com/jattach/jattach/releases/download/v2.1/jattach
    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                   Dload  Upload   Total   Spent    Left  Speed
    0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 20.205.243.166:443...
  * Connected to github.com (20.205.243.166) port 443 (#0)
  * ALPN, offering h2
  * ALPN, offering http/1.1
  * successfully set certificate verify locations:
  *  CAfile: /etc/ssl/certs/ca-certificates.crt
  *  CApath: /etc/ssl/certs
  } [5 bytes data]
  * TLSv1.3 (OUT), TLS handshake, Client hello (1):
  } [512 bytes data]
  * TLSv1.3 (IN), TLS handshake, Server hello (2):
  { [122 bytes data]
  * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
  { [19 bytes data]
  * TLSv1.3 (IN), TLS handshake, Certificate (11):
  { [2459 bytes data]
  * TLSv1.3 (IN), TLS handshake, CERT verify (15):
  { [79 bytes data]
  * TLSv1.3 (IN), TLS handshake, Finished (20):
  { [36 bytes data]
  * TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
  } [1 bytes data]
  * TLSv1.3 (OUT), TLS handshake, Finished (20):
  } [36 bytes data]
  * SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
  * ALPN, server accepted to use h2
  * Server certificate:
  *  subject: C=US; ST=California; L=San Francisco; O=GitHub, Inc.; CN=github.com
  *  start date: Feb 14 00:00:00 2023 GMT
  *  expire date: Mar 14 23:59:59 2024 GMT
  *  subjectAltName: host "github.com" matched cert's "github.com"
  *  issuer: C=US; O=DigiCert Inc; CN=DigiCert TLS Hybrid ECC SHA384 2020 CA1
  *  SSL certificate verify ok.
  * Using HTTP2, server supports multi-use
  * Connection state changed (HTTP/2 confirmed)
  * Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
  } [5 bytes data]
  * Using Stream ID: 1 (easy handle 0x55cc914ed560)
  } [5 bytes data]
  > GET /jattach/jattach/releases/download/v2.1/jattach HTTP/2
  > Host: github.com
  > user-agent: curl/7.74.0
  > accept: */*
  >
  { [5 bytes data]
  * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
  { [57 bytes data]
  * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
  { [57 bytes data]
  * old SSL session ID is stale, removing
  { [5 bytes data]
  * Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
  } [5 bytes data]
  < HTTP/2 302
  < server: GitHub.com
  < date: Wed, 13 Dec 2023 01:36:02 GMT
  < content-type: text/html; charset=utf-8
  < vary: X-PJAX, X-PJAX-Container, Turbo-Visit, Turbo-Frame, Accept-Encoding, Accept, X-Requested-With
  < location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/54976015/b8f92250-3270-42cb-bf73-004a9eb4233e?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20231213%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20231213T013602Z&X-Amz-Expires=300&X-Amz-Signature=35bc8eaa900ac77f4875fbfc757592331025692f19d543e0e69f3c4c3342c530&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=54976015&response-content-disposition=attachment%3B%20filename%3Djattach&response-content-type=application%2Foctet-stream
  < cache-control: no-cache
  < strict-transport-security: max-age=31536000; includeSubdomains; preload
  < x-frame-options: deny
  < x-content-type-options: nosniff
  < x-xss-protection: 0
  < referrer-policy: no-referrer-when-downgrade
  < content-security-policy: default-src 'none'; base-uri 'self'; child-src github.com/assets-cdn/worker/gist.github.com/assets-cdn/worker/; connect-src 'self' uploads.github.com www.githubstatus.com collector.github.com raw.githubusercontent.com api.github.com github-cloud.s3.amazonaws.com github-production-repository-file-5c1aeb.s3.amazonaws.com github-production-upload-manifest-file-7fdce7.s3.amazonaws.com github-production-user-asset-6210df.s3.amazonaws.com cdn.optimizely.com logx.optimizely.com/v1/events api.githubcopilot.com objects-origin.githubusercontent.com *.actions.githubusercontent.com wss://*.actions.githubusercontent.com productionresultssa0.blob.core.windows.net/ productionresultssa1.blob.core.windows.net/ productionresultssa2.blob.core.windows.net/ productionresultssa3.blob.core.windows.net/ productionresultssa4.blob.core.windows.net/ productionresultssa5.blob.core.windows.net/ productionresultssa6.blob.core.windows.net/ productionresultssa7.blob.core.windows.net/ productionresultssa8.blob.core.windows.net/ productionresultssa9.blob.core.windows.net/ github-production-repository-image-32fea6.s3.amazonaws.com github-production-release-asset-2e65be.s3.amazonaws.com insights.github.com wss://alive.github.com; font-src github.githubassets.com; form-action 'self' github.com gist.github.com objects-origin.githubusercontent.com; frame-ancestors 'none'; frame-src viewscreen.githubusercontent.com notebooks.githubusercontent.com support.github.com; img-src 'self' data: github.githubassets.com media.githubusercontent.com camo.githubusercontent.com identicons.github.com avatars.githubusercontent.com github-cloud.s3.amazonaws.com objects.githubusercontent.com secured-user-images.githubusercontent.com/ user-images.githubusercontent.com/ private-user-images.githubusercontent.com opengraph.githubassets.com github-production-user-asset-6210df.s3.amazonaws.com customer-stories-feed.github.com spotlights-feed.github.com objects-origin.githubusercontent.com *.githubusercontent.com; manifest-src 'self'; media-src github.com user-images.githubusercontent.com/ secured-user-images.githubusercontent.com/ private-user-images.githubusercontent.com github-production-user-asset-6210df.s3.amazonaws.com gist.github.com; script-src github.githubassets.com; style-src 'unsafe-inline' github.githubassets.com; upgrade-insecure-requests; worker-src github.com/assets-cdn/worker/ gist.github.com/assets-cdn/worker/
  < content-length: 0
  < x-github-request-id: 6F1A:13E145:136CA4:156175:65790AA0
  <
  { [0 bytes data]
    0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  * Connection #0 to host github.com left intact

2.通过在curl -O 中增加 -L 参数保证curl能够自动跟踪和请求远端返回的重定向地址

curl命令默认是不跟踪重定向地址的,通过增加 -L 参数可以保证自动跟踪重定向地址,默认可以跟踪次数为50次

如果想调整重定向跟踪次数可以通过--max-redirs参数指定

curl-L-office.jpg

如下展示了增加 -L 参数后,成功下载了jattach文件

# curl -O -L https://github.com/jattach/jattach/releases/download/v2.1/jattach
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                               Dload  Upload   Total   Spent    Left  Speed
0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 24850  100 24850    0     0     95      0  0:04:21  0:04:19  0:00:02  6454

查看下载的文件如下

curl-ls-download.jpg

3.结论

在实际项目中经常遇到接口响应超时、报错等异常场景,curl命令可以很方便的帮助我们进行接口的定位,可以说是广大 程序员的比较趁手的小工具,但是curl命令默认不会跟踪重定向地址,若想要curl跟踪重定向地址需要使用 -L 参数, 默认使用 -L时最大的跟踪次数为50次,正常情况下是足够用的了,如果想单独指定跟踪次数,可以通过使用 --max-redirs 指定。