Does anyone have a solution to make a call to http.request a blocking/synchronous call? I’d like to wait for the success function to complete before proceeding with code after the http request. If not, has anyone set up a notification message from success, or a way to poll of the request data has been received?
Thanks!
@iam3o5am Just set a variable when you do the request and don’t reset that variable until you receive the success. As long as that variable is set, you don’t do any processing of the data. Do a forum search for http request and see what you find. There’s a lot of info for it.
@iam3o5am Here’s some code that downloads 5 mp3 files. I have the function that saves the files commented out since I don’t know if you want that or not or you just want to see how to do a request and wait for a completion before starting the next download. Theses are large files, so it takes a few seconds to download them. Just tap the screen to start the download.
displayMode(FULLSCREEN)
function setup()
rectMode(CENTER)
downloadedTable={}
sizeTable={}
queTable={
{"http://www.amigaremix.com/files/2894/laki_-_Shadow_of_the_Beast_-_Death_Scene.mp3","Death_Scene"},
{"http://www.amigaremix.com/files/2910/EXON_-_Lotus_III_Theme_.mp3","Lotus_III_Theme"},
{"http://www.amigaremix.com/files/2869/Darkman007_-_Dazzler_Cover.mp3","Dazzler_Cover"},
{"http://www.amigaremix.com/files/2212/Cisskey_-_Turrican_2_-_In_the_water.mp3","In_the_water"},
{"http://www.amigaremix.com/files/2010/Sven_Storm_-_Firehawk_Loader.mp3","Firehawk_Loader"}, }
end
function draw()
background(0)
fill(255,0,0)
text("To download",WIDTH/2,HEIGHT-40)
text("Downloaded",WIDTH/2,HEIGHT-300)
text("Tap screen to start download",WIDTH/2,200)
fill(255)
for z=1,#queTable do
text(queTable[z][2],WIDTH/2,HEIGHT-30*z-60)
end
if #queTable>0 and start then
if not downloading then
downloading=true
http.request(queTable[1][1],didLoadMusic,notFound)
end
text("Downloading -- "..queTable[1][2],WIDTH/2,100)
end
for a,b in pairs(downloadedTable) do
text(string.format("%s %d bytes",b,sizeTable[a]),WIDTH/2,HEIGHT-320-a*30)
end
end
function touched(t)
if t.state==BEGAN then
start=true
end
end
function notFound()
print("error with file..."..queTable[pos][1])
downloading=false
table.remove(queTable,1)
end
function didLoadMusic(data,status,headers)
writeFile(queTable[1][2]..".mp3",data)
downloading=false
table.insert(downloadedTable,queTable[1][2])
table.insert(sizeTable,#data)
table.remove(queTable,1)
end
function writeFile(fileName, data)
--[[
local file = os.getenv("HOME").."/Documents/Dropbox.assets/"..fileName
wFd, err = io.open(file, "w")
if wFd == nil then
print("Error creating file "..fileName.."\
\
Error: "..err)
else
wFd:write(data)
wFd:close()
end
--]]
end
Thanks @dave1707 for the sample code. I suppose I’ve been simplistic in my thinking. I was hoping there was a way to just prevent execution from continuing to the line after the http.request() call, and instead rely on continuing execution from the http.request success function. Instead it looks like I just have to use some logic to loop around and control were execution happens until I get my requested data.
@iam3o5am The way Codea works is it keeps on executing. If you don’t want something to run, you skip around it. Once you do your request, you don’t execute the request again until you received what you requested or after a certain time limit you set. Once you get used to how Codea runs, it is easy to code things like that. I remember when I coded in basic, there were pause or delay commands that stopped the execution at that point until you wanted to continue. Commands like that would never work with Codea because it needs to constantly run.
You can make a blocking api yourself (or a “pool” like you mentioned). The way you do this, is you queue your function calls as coroutines and then let the coroutines work one after another by checking whether the current routine done its job or not and the continue with the next.
I posted an implementation some time ago. Please see here: https://codea.io/talk/discussion/8215/cutscenes-with-coroutines
Along the same vein, here’s some code:
function setup()
print(coroutine.running())
function callback(data)
print(coroutine.running())
end
co = coroutine.create(
function()
print(coroutine.running())
http.request("http://www.google.com/home", callback, callback)
end
)
coroutine.resume(co)
end
The output is:
thread: 0x1281adc08 true
thread: 0x1c41bacc8 false
I put callback() as both the success and fail arguments, to be sure I catch a call from either. Problem is, nothing prints from that function. The main thread resumes after ouputting what you see above.
I suppose the second return value (boolean) is whether the thread is running, though PIL does not mention it. Yet, if I add more code to the coroutine function (say, before the http.request call) then that code executes.
So my questions are:
Thanks.
@se24vad Thank you for the link. Between your code and that of @dave1707, I am getting a good picture of this approach. I have a question about your thread queue in particular:
Could you show how you would write a function, called by exec(), that itself would call Codea’s http.request() function? In other words, if I have this function:
function request()
local function success(data)
print(data)
end
http.request("http://www.google.com/home", success)
end
And then add it to your thread queue:
exec(request)
Then the request function finishes not after success() is called, but as soon as the http.request() line is complete. The coroutnie yields and I never get the data printed in the success() function.
I would love to have Codea’s http.request() execute in another coroutine and not yield until the success function is called and complete.
On that note, does anyone know how the http.request function works under the hood?
@iam3o5am I don’t know about under the hood, but once you call the request function, it does it’s own thing until it’s done. When it’s done, it calls the success function you set. So once your request starts running, Codea continues to run your regular code until the success function is called. It then runs the success code until it’s complete. Then Codea continues with your regular code.
Edit: I don’t think the request function will work if the coroutine it’s in doesn’t yield. I might have an example of a coroutine that will start a request. When the coroutine ends, the request function executes. Once the request is successful, it starts another coroutine that runs some code, yields to Codea, Codea yields to the coroutine, etc. until that coroutine exits. Then Codea runs without yielding. I don’t know if your after something like that.
I ask about under the hood, because as my example above shows (two comments up, where I use the coroutine.running() call) there are cases when the success callback function doesnt get called. It either happens when I use http.request in a separate coroutine, or if I use it in the main thread but i enter a loop and it appears that the success function is never given a chance to run. I wanted to loop and just check if I got my data yet, but the while loop never allowed success() to execute.
My guess is that @Simeon and team are using some sort of multithreaded C under the hood to handle the actual http request? I just don’t understand how that plays into the execution path that the Lua interpreter is going through…
@dave1707 In your music download code above, what if you had follow-up code to execute - where would you put that? In other words:
Thanks for being patient with me…not sure why I’m having such a mental block with this one. My ultimate goal is a general solution for blocking http.requests. In the middle of other code I may want to make some request and I want to hold everything else until I get what I want and then move on.
@iam3o5am Without knowing exactly what you’re doing, here’s something simple. The first sprite tree runs all the time. The second sprite Observatory runs only during a download. The third sprite Explosion runs after the download completed. Tap the screen to start a download. Not sure if this is what you’re after.
function setup()
x1,y1=WIDTH/2,HEIGHT/2
xv1,yv1=2,3
x2,y2=WIDTH/2,HEIGHT/2
xv2,yv2=-3,2
x3,y3=WIDTH/2,HEIGHT/2
xv3,yv3=3,-3
fill(255)
str="tap screen to start download"
end
function draw()
background(0)
text(str,WIDTH/2,HEIGHT-100)
firstSprite()
if runSecond then
secondSprite()
end
if runThird then
thirdSprite()
end
sprite("Small World:Tree Round",x1,y1)
sprite("Small World:Observatory",x2,y2)
sprite("Small World:Explosion",x3,y3)
end
function firstSprite()
x1=x1+xv1
y1=y1+yv1
if x1<0 or x1>WIDTH then
xv1=-xv1
end
if y1<0 or y1>HEIGHT then
yv1=-yv1
end
end
function secondSprite()
x2=x2+xv2
y2=y2+yv2
if x2<0 or x2>WIDTH then
xv2=-xv2
end
if y2<0 or y2>HEIGHT then
yv2=-yv2
end
end
function thirdSprite()
x3=x3+xv3
y3=y3+yv3
if x3<0 or x3>WIDTH then
xv3=-xv3
end
if y3<0 or y3>HEIGHT then
yv3=-yv3
end
end
function touched(t)
if t.state==BEGAN and not downloading then
runSecond=true
runThird=false
print("http request started\
run secondSprite")
http.request("http://www.amigaremix.com/files/2894/laki_-_Shadow_of_the_Beast_-_Death_Scene.mp3",success,fail)
downloading=true
end
end
function success(data)
print("http request ended\
run thirdSprite")
runSecond=false
runThird=true
downloading=false
end
function fail()
print("http failed")
end
@iam3o5am The http.request is probably a coroutine in itself because you can run a request without it stopping the normal Codea code. If you do a request that loads a lot of data that you need to process, then the processing code should be put in a coroutine. That way you can process the data without interrupting the normal Codea code and when you’re done with the process, then you can do whatever you want with the processed data.
@iam3o5am Here’s another example. A sprite moves around the screen. When you tap the screen to start the http.request, the sprite continues to move. When the request is complete, a process routine starts. The non coroutine process stops the sprite because it’s taking all of the cpu process. When you do the request with the runCoroutine slider on, the coroutine process will run when the request is complete. The coroutine is set up to yield every 100000 count to allow the sprite to continue to move. The coroutine will take longer, but it won’t completely stop the sprite. Changing the 100000 count to a larger or smaller number will vary the time the process routine run, but it will also vary the movement of the sprite.
function setup()
parameter.boolean("runCoroutine",false)
x1,y1=WIDTH/2,HEIGHT/2
xv1,yv1=2,3
fill(255)
limit=100000000
str="tap screen to start download"
end
function draw()
background(0)
text(str,WIDTH/2,HEIGHT-100)
firstSprite()
sprite("Small World:Tree Round",x1,y1)
if done then
done=false
process() -- run the non coroutine function
end
if yy then
coroutine.resume(yy) -- run the coroutine process
end
end
function firstSprite()
x1=x1+xv1
y1=y1+yv1
if x1<0 or x1>WIDTH then
xv1=-xv1
end
if y1<0 or y1>HEIGHT then
yv1=-yv1
end
end
function process()
print("loop process started")
print("about 25 second delay")
a=0
for z=1,limit do
a=a+math.sin(z)
end
print("loop process ended")
print("total",a)
end
function processCo()
print("coroutine loop process started")
print("about 33 second delay")
a=0
for z=1,limit do
a=a+math.sin(z)
if z%100000==0 then
coroutine.yield(yy)
end
end
yy=nil
print("coroutine loop process ended")
print("total",a)
end
function touched(t)
if t.state==BEGAN and not downloading then
output.clear()
print("http request started")
http.request("http://www.amigaremix.com/files/2894/laki_-_Shadow_of_the_Beast_-_Death_Scene.mp3",success,fail)
downloading=true
end
end
function success(data)
print("http request ended")
print("data size ",#data)
downloading=false
if runCoroutine then
print("coroutine process started")
yy=coroutine.create(processCo)
coroutine.resume(yy)
else
print("non coroutine process started")
done=true
end
end
function fail()
print("http failed")
end
@iam3o5am you can’t execute the success function BEFORE you sent a request. So if I understand you right, then what you want is to fire the success function right after the request received data. then later after success is ready, you want call some other routine. - if so then here’s how I would do it.
The key here, is that you call your functions through the exec() call, to queue them up and run after each other (in sequence). Because if you were to call it the Codea-way then success is like a coroutine and will fire only when it got data. By that time your other functions will be already called - thus will run BEFORE the success function, which is what you try to avoid as I understand.
local function success(response)
while not response do -- block until data from http request comes in
coroutine.yield()
end
print(response) -- when data arrived, processing it
print("response received. done.")
end
local function request()
http.request("http://google.com/home", success)
print("request sent. wait...")
end
local function continue_with_some_other_stuff()
print("now that we completed the request and processed the response, it's time to continue with the rest of the queue")
end
function setup()
request() -- start request
exec(sucess) -- create a queue to work through
exec(continue_with_some_other_stuff)
end
function draw()
thread_update()
end
@dave1707 Thanks for your recent post - it did get me over a few obstacles in my thinking - especially your done variable. @se24vad, also - thank you, though I still have some issues with the code you posted. If I flesh it out with an exec and thread_update functions (from your previous code, then the print order is still not what I expect/hope for. Here’s the full code:
local thread_queue = {}
local function exec(func, ...)
local params = {...}
local thread = function(self) func(self, unpack(params)) end
table.insert(thread_queue, coroutine.create(thread))
end
local function thread_update()
if #thread_queue > 0 then
if coroutine.status(thread_queue[1]) == "dead" then table.remove(thread_queue, 1)
else coroutine.resume(thread_queue[1], thread_queue[1]) end
end
end
function draw()
thread_update()
end
local function success(response)
while not response do -- block until data from http request comes in
coroutine.yield()
end
print(response) -- when data arrived, processing it
print("response received. done.")
end
local function request()
http.request("http://google.com/home", success)
print("request sent. wait...")
end
local function continue_with_some_other_stuff()
print("now that we completed the request and processed the response, it's time to continue with the rest of the queue")
end
function setup()
request() -- start request
exec(success) -- create a queue to work through
exec(continue_with_some_other_stuff)
end
Here’s the response order I get:
request sent. wait...
thread: 0x1c01bb9e8
response received. done.
now that we completed the request and processed the response, it's time to continue with the rest of the queue
<!DOCTYPE html>
<title>Google</title>
<script>
var url = 'https://madeby.google.com/home/';
url += window.location.search;
url += window.location.hash;
window.location = url;
</script>
response received. done.
Reflection:
The response data (arbitrary google.com request) comes back after the exec() calls are made. In order words, when Insee “Response received. Done.”, I would hope to have the google data already printed.
The exec(success) call causes the thread ID to print, again before getting the google data.
The goal is for the print statements to be in this order:
Thanks for everyone throwing code my way! Every snippet helps.
In fact, that’s the challenge. (“Programming challenge! everyone join in!”). I would love to see the shortest code snippet (just the essentials, so its easy to see the blocking logic) that can:
Nothing else. 3 MUST wait until 2 prints its response data.
The drawing or other messages in some of the examples makes things more interesting, but also more complex and clouds the essential blocking logic. I just want to understand steps 1 - 3 in their purest form.
Thanks again.
oh, that’s really weird. Here’s my complete test code with a screenshot of the final result I get on my iPad Pro.
local thread_queue = {}
local function exec(func, ...)
local params = {...}
local thread = function(self) func(self, unpack(params)) end
table.insert(thread_queue, coroutine.create(thread))
end
local function thread_update()
if #thread_queue > 0 then
if coroutine.status(thread_queue[1]) == "dead" then table.remove(thread_queue, 1)
else coroutine.resume(thread_queue[1], thread_queue[1]) end
end
end
local function success(response)
while not response do -- block until data from http request comes in
coroutine.yield()
end
print(response) -- when data arrived, processing it
print("response received. done.")
end
local function request()
http.request("http://google.com/home", success)
print("request sent. wait...")
end
local function continue_with_some_other_stuff()
print("now that we completed the request and processed the response, it's time to continue with the rest of the queue")
end
function setup()
request() -- start request
exec(sucess) -- create a queue to work through
exec(continue_with_some_other_stuff)
end
function draw()
thread_update()
end
@iam3o5am Here’s the code to do your 4 steps above. You never said what you’re doing before you want to do the request, or what you want to do with the data received, or how many request you want to do, or what you want to do when the request is done. It doesn’t take a lot of code to do the request, but what else you want to do will change the code below. It’s easy to write small code that does one thing, but it will take more code to make it work with the other code you don’t mention.
function setup()
request()
end
function request()
print("request sent")
http.request("http://google.com/home", success)
end
function success(data)
print(data)
print("response received")
end
@dave1707 - its not that I am holding back on what I am trying to do. I am just working on the concept of a blocking call. I just dont see it yet. The code you posted relies on the success function for the final (‘im moving on step’). The only reason this works is because there is nothing left in the setup function. But this isn’t realistic. In any larger program there will be more to do where the original call was made (setup in this case, but likely somewhere else in a larger program). In my mind, setup is just a callback to handle the response - not where the main code path would be.
I apologize if Im coming across as frustrating on not communcating my intention, or now I suppose I could be accused of first wanting small code, and now saying something is not realistic and wouldnt be that way in a larger program. I recognize that, and so possibly my error is in my thinking. I need to change how I am approaching the problem.
That said, what I mean by my 3-step goal is:
To demo that NOT working:
function setup()
request()
print("continue")
end
function request()
print("request sent")
http.request("http://google.com/home", success)
end
function success(data)
print(data)
print("response received")
end
The “continue” line is out of order above.
Now, that is obvious to all of us. That is why everyone gave their original, longer suggestions. Either way, thats what I would like to solve in its purest form / shortest code.
@dave1707 I do appreciate your contributions and I apologize again if this has been a frustrating discussion thread for you. Please dont feel obligated to respond if I keep hammering at this. Maybe the bottom line is, there isn’t a simple/pure/short way to block and that it requires looping around and checking variables and its implmentation is case by case specific.
@se24vad I haven’t look yet at your updated post. On it now…
@se24vad Yours seems to work, but its because ‘success’ is misspelled in the exec(success) call. You have it as ‘sucess’.