HTTP request 'HEAD' method not working as expected

Questions
1

Hi all,

I’m hoping I’m totally off the mark here and there’s an easy solution but I can’t seem to get my http request to only return the headers in it’s response. The docs seem to imply it can be done but I’m still receiving content as well. I’m using a request to grab the ETag to detect when a file has changed (and only if it has changed I download it fully).

local function GetServerEtag(url, cb)
    local success = function(data, status, head)
        if status == 200 then
            if data ~= nil then
                print("HEAD request returned some data! This shouldn't happen:\
\
" .. data)
            end
            cb(head["Etag"])
        else
            cb(nil)
        end
    end
    
    local fail = function(error)
        cb(nil)
    end
    
    local params = { ["method"] = "HEAD" }
    
    http.request(url, success, fail, params)
end

The error message is always displayed followed by the file content (I’ve tried multiple urls). I’m not sure if Codea or iOS is doing some sort of caching of random urls in the background itself but I suspect not.

Any advice would be much appreciated!

Thanks,
Steppers

2

Good question, I had to run your example through the debugger to see what was happening

It looks like the data in this case is not being fetched from the server, but it is also non-nil, it is a special NSZeroData type that is simply 0 bytes long and comes back with the HEAD request. So it looks like your code is simply doing a HEAD request and no data is being fetched but we are returning a 0-byte data object instead of nil to your callback

A check for:

if data ~= nil and data ~= "" then

Should catch this

3

Thanks for getting back to me.

I’m not sure what’s going on but there’s definitely data being printed out xD

local function GetServerEtag(url, cb)
    local success = function(data, status, head)
        if status == 200 then
            if data ~= nil and data ~= "" then
                print("[ERROR] HEAD request returned data! This shouldn't happen\
\
" .. data)
            end
            cb(head["Etag"])
        else
            cb(nil)
        end
    end
    
    local fail = function(error)
        cb(nil)
    end
    
    local params = { ["method"] = "HEAD" }
    
    http.request(url, success, fail, params)
end

function setup()
    GetServerEtag("https://www.w3.org/TR/PNG/iso_8859-1.txt", function(etag)
        print(etag)
    end)
end

That should replicate the exact issue I’m having with a test file. I’m just hoping it’s not the server configuration causing problems…

4

Strange, I ran your code and do not get any data printed. The data returned is the empty string ""

5

Ok, odd. I’ll try a few more things then to try to narrow down what’s going on.

6

So it seems either iOS or Codea (I suspect iOS) is caching the url result (when I actually download it) for the duration of the App running.

  1. Close & Reopen Codea
  2. Run the example I posted above and view desired result.
  3. Without closing Codea, run the code below which should run the same etag retrieval function before downloading the file fully.
  4. Without closing Codea, run the example above again which should reproduce the problem I am describing.

If I close and reopen the app after downloading the file the data is as you explain just the empty string.

To add to it, if I disable all internet connections on my device after downloading the file, the initial example still returns data. So yeah I think it’s just iOS and if I have no connection I don’t think it’ll be wasting data.

local function DownloadFile(url, file, cb)
    print("Downloading...")
    local success = function(data, status, head)
        if status == 200 then
            if type(data) == "userdata" then
                saveImage(file, data)
            else
                saveText(file, data)
            end
            cb(data)
        else
            cb(nil)
        end
    end
    
    local fail = function(error)
        cb(nil)
    end
    
    http.request(url, success, fail)
end

function setup()
    DownloadFile("https://www.w3.org/TR/PNG/iso_8859-1.txt", asset.documents .. "test.txt", function(data)
        if data ~= nil then
            print("Success")
        else
            print("Failure")
        end
    end)
end
7

I tried a little program I had and the results I got was the first time I ran the code, the data size was 0. When I ran it without the params and got the data, no matter what I did after that, I always got all the data. So yes, somewhere the data is cached.

function setup()
    url="https://www.w3.org/TR/PNG/iso_8859-1.txt"
    params={ ["method"] = "HEAD" }
    http.request(url, success, fail, params)
end
    
function success(data,status,head)
    print("success")
    print(#data)
    print(status)
    print(head)  
end

function fail(data,status,head) 
    print("fail") 
    print(data)
    print(status)
    print(head)
end
8

Thanks for confirming my suspicions.

I suspect it’s a small optimisation iOS makes that probably helps some web apps cut down on data use a fair bit. Good to know what’s going on now though.