HTTP Fundamentals

HyperText Transfer Protocol (HTTP)

URL

HTTP Flow

cURL

# curl inlanefreight.com
# curl -O inlanefreight.com/index.html  // -O:使用cURL下载页面或文件，并使用标志将内容输出到文件中  // -o:指定输出文件名
# curl -s -O inlanefreight.com/index.html  // -s:静默状态

Hypertext Transfer Protocol Secure (HTTPS)

Man-in-the-middle(MiTM)

HTTPS Overview

Note:  Although the data transferred through the HTTPS protocol may be encrypted, the request may still reveal the visited URL if it contacted a clear-text DNS server. For this reason, it is recommended to utilize encrypted DNS servers (e.g. 8.8.8.8 or 1.2.3.4), or utilize a VPN service to ensure all traffic is properly encrypted.

HTTPS Flow

Note:  Depending on the circumstances, an attacker may be able to perform an HTTP downgrade attack, which downgrades HTTPS communication to HTTP, making the data transferred in clear-text. This is done by setting up a Man-In-The-Middle (MITM) proxy to transfer all traffic through the attacker's host without the user's knowledge. However, most modern browsers, servers, and web applications protect against this attack.

key exchange

SSL certificates exchange

cURL for HTTPS

    # curl -k https://inlanefreight.com
    
    // -k: We may face such an issue when testing a local web application or with a web
    //application hosted for practice purposes, as such web applications may not yet have 
    //implemented a valid SSL certificate. To skip the certificate check with cURL, we can 
    //use the `-k` flag
    
  

HTTP Requests and Responses

HTTP Request

HTTP Response

cURL

    # curl inlanefreight.com -v //-v:查看完整的HTTP请求与相应
    
    

Note:  The -vvv flag shows an even more verbose output. Try to use this flag to see what extra request and response details get displayed with it.

HTTP Headers

General Headers

Entiry Headers

Request Headers

Response Headers

Security Headers

cURL

    # curl -I https://www.inlanefreight.com   //-I:只显示响应头,不显示正文
    # curl https://www.inlanefreight.com -A 'Mozilla/5.0'   //-A：设置User-Agent
    

Note: In addition to viewing headers, cURL also allows us to set request headers with the -H flag, as we will see in a later section. Some headers, like the User-Agent or Cookie headers, has their own flags. For example, we can use the -A to set our User-Agent

Browser DevTools

HTTP Methods

HTTP Methods and Codes

Request Methods

Response Codes

GET

HTTP Basic Auth

Note: Unlike the usual login forms, which utilize HTTP parameters to validate the user credentials (e.g. POST request), this type of authentication utilizes a basic HTTP authentication, which is handled directly by the webserver to protect a specific page/directory, without directly interacting with the web application.

    # curl -i http://<SERVER_IP>:<PORT>/
    //-i: 查看响应标头与响应正文
    # curl -u admin:admin http://<SERVER_IP>:<PORT>/ 
    //-u：提供 basic HTTP authentication 的凭证
    
    # curl http://admin:admin@<SERVER_IP>:<PORT>/
    
    

HTTP Authorization Header

    # curl -H 'Authorization: Basic YWRtaW46YWRtaW4=' http://<SERVER_IP>:<PORT>/
    //-H ：设置标头

GET Parameters

POST

Login Forms

  # curl -X POST -d 'username=admin&password=admin' http://<SERVER_IP>:<PORT>/
  
  

Tip:  Many login forms would redirect us to a different page once authenticated (e.g. /dashboard.php). If we want to follow the redirection with cURL, we can use the -L flag.

Authenticated Cookies

    # curl -X POST -d 'username=admin&password=admin' http://<SERVER_IP>:<PORT>/ -i
    # curl -b 'PHPSESSID=c1nsa6op7vtk7kdis7bcnbadf1' http://<SERVER_IP>:<PORT>/
    # curl -H 'Cookie: PHPSESSID=c1nsa6op7vtk7kdis7bcnbadf1' http://<SERVER_IP>:<PORT>/
    
    

As we can see, having a valid cookie may be enough to get authenticated into many web applications. This can be an essential part of some web attacks, like Cross-Site Scripting.

JSON Data

#  curl -X POST -d '{"search":"london"}' -b 'PHPSESSID=c1nsa6op7vtk7kdis7bcnbadf1' -H 'Content-Type: application/json' http://<SERVER_IP>:<PORT>/search.php

As we can see, we were able to interact with the search function directly without needing to login or interact with the web application front-end. This can be an essential skill when performing web application assessments or bug bounty exercises, as it is much faster to test web applications this way.

CRUD API

APIs

    # curl -X PUT http://<SERVER_IP>:<PORT>/api.php/city/london ...SNIP...
    

CRUD

Read

    # curl http://<SERVER_IP>:<PORT>/api.php/city/london
    

    # curl -s http://<SERVER_IP>:<PORT>/api.php/city/london | jq // jq:格式化json数据
    

    # curl -s http://<SERVER_IP>:<PORT>/api.php/city/ | jq  // we can pass an empty string to retrieve all entires in the table
    

Create

    # curl -X POST http://<SERVER_IP>:<PORT>/api.php/city/ -d '{"city_name":"HTB_City", "country_name":"HTB"}' -H 'Content-Type: application/json'
    
    

Update

Note:  The HTTP PATCH method may also be used to update API entries instead of PUT. To be precise, PATCH is used to partially update an entry (only modify some of its data "e.g. only city_name"), while PUT is used to update the entire entry. We may also use the HTTP OPTIONS method to see which of the two is accepted by the server, and then use the appropriate method accordingly. In this section, we will be focusing on the PUT method, though their usage is quite similar.

    # curl -X PUT http://<SERVER_IP>:<PORT>/api.php/city/london -d '{"city_name":"New_HTB_City", "country_name":"HTB"}' -H 'Content-Type: application/json'
    
    

注意： 在某些 API 中，该Update操作也可用于创建新条目。基本上，我们会发送我们的数据，如果它不存在，它会创建它。例如，在上面的示例中，即使带有london城市的条目不存在，它也会使用我们传递的详细信息创建一个新条目。然而，在我们的示例中，情况并非如此。尝试更新一个不存在的城市，看看你会得到什么。

Delete

    # curl -X DELETE http://<SERVER_IP>:<PORT>/api.php/city/New_HTB_City