How to download a CGI-generated GIF-image


In the Arduino → Examples → Ethernet you find the WebClient.ino that fetches a Google-search for the item “Arduino”. While this is a good start there is still some work to be done.

Of course the first thing to do is make sure the settings match your local net (MAC address, IP address, …).

Assume the counter you want to read is accessed by this URL:

http://counter.SERVERNAME.net/cgi-bin/Count.cgi?dd=E&df=CUSTOMER&ft=1
(At least the parts printed in capital letters are to be replaced by the ones for your counter.)

When you replace “www.google.com” by “counter.SERVERNAME.net” and “/search?q=arduino” by “/cgi-bin/Count.cgi?dd=E&df=CUSTOMER&ft=1” (or whatever your counter is accessed by) in the above named example the results will be astonishing and disappointing; as the CGI server will not know the amount of data to be sent it has to use “chunked transfer encoding”. In the header you can read: “Transfer encoding: chunked”

The result will be it fills plenty of CR/LF in the file. This is what I got:

0090 36 0D 0A 47 49 46 38 39 61 0D 0A 32 0D 0A 24 00 6..GIF89a..2..$.
00A0 0D 0A 32 0D 0A 09 00 0D 0A 31 0D 0A F0 0D 0A 31 ..2......1.....1
00B0 0D 0A 00 0D 0A 31 0D 0A 00 0D 0A 31 0D 0A 00 0D .....1.....1....
(I made Arduino to produce a DEBUG-like output.)

And the algorithm for decoding chunked transfer is far from being simple; you cannot just ignore all “0D/0A” because some of them could and will be real data.

But there is a way to get rid of all this trouble: in the request set the HTTP version to “1.0” instead of “1.1”. In the older version, the chunked encoding did not exist.

After doing this you will get:

0060 2D 54 79 70 65 3A 20 69 6D 61 67 65 2F 67 69 66 -Type:.image/gif
0070 0D 0A 0D 0A 47 49 46 38 39 61 24 00 09 00 F0 00 ....GIF89a$.....
0080 00 00 00 00 FF FF FF 21 FE 69 43 6F 75 6E 74 2E .......!.iCount.
with no inserted “0D/0A”s.

Now you can either store the data in RAM or on SD card or process them immediately as shown in my article “GIF-decode with Arduino –can it be done?”

By the way: all this takes less than half of the FLASH and half of the SRAM of the Arduino-UNO.

Hardware requirements

  • Ethernet Shield
  • any Arduino capable to host the Ethernet shield
  • Software requirements

  • the source



  • contact: nji(at)gmx.de