Anil's Blog

Basics of Curl

Anil Vaitla's photo
Anil Vaitla
·

4 min read

It's good to know how curl works. You don't need to know much. The flags -X (the http method to use, -XGET or -XPOST), -H (any headers to include - to add multiple just do it multiple times - curl -H"Key1: Val1" -H"Key2: Val2"), -d to put a raw payload, and -v (print all headers in verbose mode). These flags get you most of what you need - and the rest you can google. You should also know how to use jq to format json and grep to extract a specific field. Here are some exercises you can do on your own to get familiar:

Basic Curl and Formatting Commands:

1 - Make a post request with a json payload (you need -X to specify POST, -H to specify Content-Type: application/json and -d for the data):

curl  -XPOST 'https://postman-echo.com/post' -d'{"key":"value"}' -H"Content-Type: application/json"

Gives output:

{"args":{},"data":{"key":"value"},"files":{},"form":{},"headers":{"x-forwarded-proto":"https","x-forwarded-port":"443","host":"postman-echo.com","x-amzn-trace-id":"Root=1-60469547-27cdf8610e4913c1720d277d","content-length":"15","user-agent":"curl/7.64.1","accept":"*/*","content-type":"application/json"},"json":{"key":"value"},"url":"https://postman-echo.com/post"}%

2 - We'd like to see the response more clearly (so we can pipe to | jq)

curl  -XPOST 'https://postman-echo.com/post' -d'{"key":"value"}' -H"Content-Type: application/json" | jq

Gives the output:

{
  "args": {},
  "data": {
    "key": "value"
  },
  "files": {},
  "form": {},
  "headers": {
    "x-forwarded-proto": "https",
    "x-forwarded-port": "443",
    "host": "postman-echo.com",
    "x-amzn-trace-id": "Root=1-60469589-7ab2266f118090355aa8c4bf",
    "content-length": "15",
    "user-agent": "curl/7.64.1",
    "accept": "*/*",
    "content-type": "application/json"
  },
  "json": {
    "key": "value"
  },
  "url": "https://postman-echo.com/post"
}

3 - We're missing the HTTP headers which are useful for debugging (so we can add -v):

curl -v -XPOST 'https://postman-echo.com/post' -d'{"key":"value"}' -H"Content-Type: application/json" | jq

Gives output:

Note: Unnecessary use of -X or --request, POST is already inferred.
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 52.6.96.161...
> POST /post HTTP/2
> Host: postman-echo.com
> User-Agent: curl/7.64.1
> Accept: */*
> Content-Type: application/json
> Content-Length: 15
> 
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
} [15 bytes data]
* We are completely uploaded and fine
< HTTP/2 200 
< date: Mon, 08 Mar 2021 21:23:24 GMT
< content-type: application/json; charset=utf-8
< content-length: 366
< vary: Accept-Encoding
< 
{ [365 bytes data]
100   381  100   366  100    15   1678     68 --:--:-- --:--:-- --:--:--  1747
* Connection #0 to host postman-echo.com left intact
* Closing connection 0
{
  "args": {},
  "data": {
    "key": "value"
  },
  "files": {},
  "form": {},
  "headers": {
    "x-forwarded-proto": "https",
    "x-forwarded-port": "443",
    "host": "postman-echo.com",
    "x-amzn-trace-id": "Root=1-604695cc-51dd15ef4a6f6a717700522b",
    "content-length": "15",
    "user-agent": "curl/7.64.1",
    "accept": "*/*",
    "content-type": "application/json"
  },
  "json": {
    "key": "value"
  },
  "url": "https://postman-echo.com/post"
}

4 - Note we can remove the -X since when have a payload (-d flag) as it is inferred. Let's also format it better and extract the url field with a grep command:

curl -XPOST 'https://postman-echo.com/post' \
     -d'{"key":"value"}' \
     -H"Content-Type: application/json" \
     -v | jq | grep '"url"'

Gives output:

  "url": "https://postman-echo.com/post"
http