Efficient way to perform HTTP cURL GET/POST Requests with Payload

When you perform an HTTP/HTTPS GET or POST methods to get a payload from the server, you can make a request for a compressed payload by passing in the --compressed option.

By doing so the response is provided to you via the network in a compressed form thus saving bandwidth, thus you receive the data quite quickly.

Example 1 : Normal HTTP GET Request

% curl -o news.html "https://news.google.com/search?q=technology%20news&hl=en-US&gl=US&ceid=US%3Aen"                
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 2686k    0 2686k    0     0  1652k      0 --:--:--  0:00:01 --:--:-- 1654k

Now let's make this request efficient by using the --compressed option.

Example 2 : HTTP GET Request with --compressed option

% curl --compressed -o news.html.gz "https://news.google.com/search?q=technology%20news&hl=en-US&gl=US&ceid=US%3Aen"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  459k    0  459k    0     0   308k      0 --:--:--  0:00:01 --:--:--  309k

As you can see in the second example, the bytes received are 459k as compared to 2686k which is ~80% less.

Note: When the --compressed option is used, cURL requests the server to provide the data in the compressed form before sending it to you, and then cURL uncompresses it before displaying it to you to Standard Output. If the server does not support compression, you will get the data without compression as usual.

Facing issues? Have Questions? Post them here! I am happy to answer!

Click To Load Comments