Objc AR Modern obstacle? (and workaround)

Questions
1

objc bridge: SIMD types come back as nil (and one escapes pcall); some structs come back as opaque NSValue

@Steppers maybe?

Codea 3.19 (652), iOS 26.2.1, iPhone, Modern runtime.

Claude and I struggled to build a Modern project that uses ARKit through objc: voxel helicopters you conjure with a hand gesture, which stay fixed in the room as you walk around them. The ARKit session runs, and the camera feed and Vision hand tracking work. But the camera’s position in the room can’t be read directly. ARKit returns it as a SIMD matrix (simd_float4x4), and the bridge returns nil for SIMD types. In one case it throws an error that pcall can’t catch.

What I found:

  • SIMD properties return nil without any error: SCNNode.simdScale, simdOrientation, simdTransform, and ARCamera.transform (simd_float3, simd_quatf and simd_float4x4).
  • ARCamera.eulerAngles (simd_float3) raises a native error that escapes pcall: it reports “caught (…) unknown error during protected_function call” from objc.call in LuaBridge.lua:33 and aborts draw(). If it’s read every frame, the traceback keeps growing (“skipping 419 levels”), and afterwards even imageResolution reads nil.
  • SceneKit structs come back as NSConcreteValue with no readable fields: SCNNode.scale (SCNVector3), orientation (SCNVector4) and transform (SCNMatrix4). By contrast, ARCamera.imageResolution (CGSize) unwraps fine, with .width and .height.

Minimal repro:

function setup()
  local n=objc.SCNNode()
  print("scale",n.scale)                -- NSConcreteValue, no fields
  print("simdTransform",n.simdTransform) -- nil
  arSession=objc.ARSession()
  arSession:runWithConfiguration_(objc.ARWorldTrackingConfiguration())
end
function draw()
  local f=arSession.currentFrame
  if f and not done then
    done=true
    local cam=f.camera
    print("imageResolution",cam.imageResolution.width) -- 1920, works
    print("transform",cam.transform)                   -- nil
    print(pcall(function() return cam.eulerAngles end)) -- error escapes pcall
  end
end

Output:

scale	<NSConcreteValue> Retained: true
simdTransform	nil
imageResolution	1920.0
transform	nil
Error in call to draw():
caught (...) unknown error during protected_function call
stack traceback:
	[C]: in function 'objc.call'
	.../LuaKit.framework/LuaBridge.lua:33: in function 'base.sendMesg'
	[C]: in function 'base.pcall'
	.../LuaKit.framework/LuaBridge.lua:39: in function 'base.objcCallMethod'
	(...tail calls...)
	Test.lua:15: in function <Test.lua:15>
	[C]: in function 'base.pcall'
	Test.lua:15: in function 'base.draw'

The error goes straight through the pcall on line 15, so a script can’t protect itself from it.

Workaround, for anyone else hitting this: tostring(camera.description) does reach Lua, and it contains the pose as text. It looks like transform=<translation=(x y z) rotation=(a° b° c°)>, along with focalLength and principalPoint. Parsing those numbers and rebuilding the matrix (yaw × pitch × roll) gives working world tracking in Modern today. That’s what my copters run on. It’s fragile, though: it depends on Apple’s debug text format and only gives two decimal places of rotation.

objc fix if possible to do maybe please?: please have the bridge convert SIMD vectors, quaternions and matrices to Codea’s vec3, vec4, quat and mat4 (or to plain number tables), and unwrap NSValue-boxed structs the way CGSize already is. ARKit returns nearly all of its geometry as SIMD types: the camera pose, anchors, raycast results. Getting just transform through would make proper world-tracked AR possible through objc until the built-in AR API arrives.