This is probably best addressed directly to @Simeon to be honest but figured any discussion could be useful to others.
I’ve been making good progress on a minimal implementation of the luarocks package manager for use in Codea but have been struggling with some strange behaviour around the require() function in the editor.
This is the section of code I’m working with (the rest isn’t easily available as a single project yet):
local vfs = VFS("luarocks")
local function rock_loader(data)
return function()
return load(data)()
end
end
local function rock_searcher(module)
local search_paths = {
"/share/lua/" .. module .. ".lua",
"/share/lua/" .. module:gsub("%.", "/") .. ".lua",
"/share/lua/" .. module .. "/init.lua"
}
-- Search the potential paths until we find a valid file
for _,path in ipairs(search_paths) do
local f = vfs:readFile(path)
if f then
return rock_loader(f)
end
end
if module == "MessagePack" then
----------------Only works with this present-----------------
-------------------------------------------------------------
return rock_loader('return { "I\\'m never actually used!" } ')
-------------------------------------------------------------
end
-- We found no rock modules
return nil
end
table.insert(package.searchers, 2, rock_searcher)
local ok, res = pcall(require, "MessagePack")
if ok then
print("pcall 'require' success:", res)
else
print("pcall 'require' fail:", res)
end
local res = require("MessagePack")
if res then
print("Plain 'require' success:", res)
else
print("Plain 'require' fail:", res)
end
The call to require wrapped in a pcall works just fine and it correctly finds the file and loads it without needing the if module == "MessagePack" then workaround.
The plain require call however, trips up immediately (without ever launching the project) without the workaround present making me wonder if something in the editor’s error checking is causing it?
Without the workaround I just see module ‘MessagePack’ not found in the editor on the local res = require("MessagePack") line. The most puzzling thing is that when the workaround is in place, it’s never actually used at all and the plain require correctly loads the intended file instead.
This has been causing me some real headaches just to get to this point so if anyone has any ideas I’d really appreciate it!
It could have something to do with the way Codea executes code outside of the setup() & draw() functions.
print("I print twice")
function setup()
print(“I print once”)
end
function draw()
background(40, 40, 50)
end
In this case I’d expect the print() at the top to only be executed once, however when launching from the editor the string prints twice. When tapping the restart button after launching it the first time it’s only printed once.
Is this due to some error checking being done before the project actually launches?
Note that the initial issue with the require() function is not present when executing from within setup.
I’ve managed to narrow down the issue to the readText function when it’s used outside of setup() & draw() functions.
Here’s a minimal repro case:
local data = readText(asset.Main)
if data == nil then
-- Errors in the editor here when uncommented as the readText call above fails
-- error(tostring(data))
end
function setup()
print("Data from outside setup():\
" .. tostring(data):sub(1, 32) .. "...")
data = readText(asset.Main)
print("Data from inside setup():\
" .. tostring(data):sub(1, 32) .. "...")
end
It seems to be related to the 2 execution passes over the script before setup() is called. The first of which seems to cause readText to fail to read the file and return nil instead for some reason. Is it not fully initialised during this first pass @Simeon?
And wrapping it in pcall means the file reads successfully somehow:
local ok, data = pcall(readText, asset.Main)
if ok then
print(tostring(data):sub(1, 32) .. "...")
else
error(tostring(data))
end
function setup()
print("Data from outside setup():\
" .. tostring(data):sub(1, 32) .. "...")
data = readText(asset.Main)
print("Data from inside setup():\
" .. tostring(data):sub(1, 32) .. "...")
end
@Steppers I’m not seeing the issue you describe for the nil readText. If I do a readText before setup() and 1 in setup(), and print each read, I print all 3. I don’t get the nil result. A restart results in 2 prints since the double pass isn’t required. The double pass when the project starts has been around for as long as I can remember. There’s very few commands that I use outside of a function because of the double pass at startup.
@dave1707 What about this one? The nil result only seems to be detectable if you generate an error manually.
This should produce the error in the nil check block:
local data = readText(asset.Main)
if data == nil then
-- Errors in the editor here when uncommented as the readText call above fails
error(tostring(data))
end
print("Data while outside setup():\
" .. tostring(data):sub(1, 32) .. "...")
function setup()
print("Data from outside setup():\
" .. tostring(data):sub(1, 32) .. "...")
data = readText(asset.Main)
print("Data from inside setup():\
" .. tostring(data):sub(1, 32) .. "...")
end
@Steppers You’re trying to read the Main tab with readText and it’s possible that the Main tab isn’t there yet to be read. Maybe it’s not there until the 2nd pass. Try running this code that calls the function “test” on the first pass. It causes an error because it doesn’t know anything about the function “test” yet.
When I was doing the readTest before setup() and got all 3 prints instead of a nil, I was reading a text file that already existed.
test()
function setup()
end
function test()
print("test")
end
I get an error in the editor reading nil | Asset Key meaning at least 1 pass is returning nil from readText. If I use the print line instead it never prints a nil value.
That must be triggering the error in the very first pass (maybe even before the 2 we can see with print()) causing the project to never launch in the first place.
This becomes a problem when trying to require modules outside of the setup() function as is common in much of standard Lua. My custom luarocks searcher for the require function relies on being able to read a json file to determine which file should be loaded but as the json is failing to load in the very first pass, require is then throwing an error meaning the project never launches at all.
@Steppers I’m wrong about the Main tab not being there on the first pass. If I run the below code, it prints the Main tab twice for the 2 passes, then once from setup.
But then I’m confused about what’s really happening if you uncomment the 2nd print. That’s where your nil error shows up. But if it printed the Main tab twice the first time then “data” should have had something in it and not be nil.
Maybe there’s more than 2 passes that we don’t know about and can’t test for.
data = readText(asset.Main)
print("===== before setup =====\
",data)
--print("before setup=====\
"..data)
function setup()
data = readText(asset.Main)
print("===== in setup =====\
"..data)
end
Though many of the standalone issues I’ve mentioned above still stand It seems the main issue in my luarocks loading code was of my own making…
I had an error call inside my virtual file system code that would trigger when we attempted to access a path that didn’t exist in the VFS. Now normally when running from the user’s code this would be just fine and the editor would display the relevant error from it, but in this case I suspect this was being triggered when an internal module middleclass and block was being required.
Unfortunately, this error call was causing Codea to crash entirely so the runtime is clearly missing some error checking at that point in the project startup sequence.
@Simeon Could be something to look into in future but I have everything working now that I was trying to do so it’s nothing urgent.
@Simeon Removing the last line should cause Codea to crash. Admittedly, raising an error in the custom module searcher does go against the lua 5.4 reference but It probably shouldn’t be crashing Codea.
local function custom_searcher(module)
if module == "HelloModule" then
return function(module, data)
return load("return 'Hello Codea'")()
end, "nothing_important"
end
-- Lua 5.4 Reference states:
-- 'Searchers should raise no errors and have no side effects in Lua'
-- So I guess this could be classified as undefined behaviour.
error("No module found in custom searcher")
-- No module found
return nil
end
table.insert(package.searchers, 2, custom_searcher)
local str = require("HelloModule")
function draw()
background(40)
end
-- Codea crashes if I remove this.
error(str)
The error in the custom searcher was causing some custom Craft modules to throw exceptions when running a project. We catch the exceptions now, so this is fixed — in that it won’t crash Codea. Though signalling an error in a custom searcher will likely cause other parts of Codea to stop working (as those modules won’t be loaded)
@Simeon Amazing, thank you. Just good to not have Codea crash.
I know the error shouldn’t be there anyway and I’ve hopefully removed anything from my code that would trigger an error like that again so from my end we’re all good.
Just seems to be silently failing.