Any chance of a key down/up API for external keyboards?
Either using GCKeyboard or UIResponder::pressesBegan(). pressesBegan is supported in iOS 13.4.
That and potentially APIs making use of GCGamepad & GCMouse (& prefersPointerLocked).
I know the hover gesture should also provide the mouse location but mouse locking would also be great.
There’s a fair bit of work involved here I know but just wanted to throw the suggestion out there.
Cheers,
Steppers
We have some (possibly undocumented) stuff that was added to support the Shade UI:
-- Mouse/trackpad was moved (no button held down)
function hover(gesture)
end
-- Mouse/trackpad was scrolled (i.e. 2 finger scroll gesture)
function scroll(gesture)
end
Where gesture has the following stuff (ignore the binding code):
LuaBinding(L)
.beginClass<Gesture>("gesture")
.addPropertyReadOnly("location", &Gesture::getLocation)
.addPropertyReadOnly("translation", &Gesture::getTranslation)
.addPropertyReadOnly("delta", &Gesture::getDeltaTranslation)
.addPropertyReadOnly("touchCount", &Gesture::getNumberOfTouches)
.addPropertyReadOnly("state", &Gesture::getState)
.addPropertyReadOnly("shift", &Gesture::shiftEnabled)
.addPropertyReadOnly("capsLock", &Gesture::capsLockEnabled)
.addPropertyReadOnly("alt", &Gesture::altEnabled)
.addPropertyReadOnly("control", &Gesture::controlEnabled)
.addPropertyReadOnly("command", &Gesture::commandEnabled)
.addPropertyReadOnly("numPad", &Gesture::numPadEnabled)
.endClass();
With location, translation, delta all being vec2 types and state using the obligatory BEGAN, MOVING, ENDED, CANCELLED constants.
The alt, control, etc. are just booleans indicating if those keys were held during the gesture.
Since iOS 14 supports proper access to mouse stuff it might be worth adding an API for it. Maybe it would look like this?
local mouse = input.mouse
-- or mouse = input.mice[1]
mouse.buttonChanged = function(mouse, button, isPressed)
print("Pressed mouse button:", button)
end
mouse.moved = function(mouse, position, delta)
end
mouse.leftButton
mouse.middleButton
mouse.rightButton
mouse.auxiliaryButtons[1]
mouse:capture()
mouse:release() -- not sure if an API for this exists yet
Codea 4 (our new Metal runtime built from the ground up) will have support for keyboard callbacks as global functions, but could also be done similar to the mouse if we use GCKeyboard.
I also want to have support for gamepads, so we can use PS4, Xbox, etc… controllers.
Brilliant,
I had seen the scroll & hover globals which should work well with the mouse in general but once the cursor is ‘captured’ the hover function will no longer receive the events as the gesture handler no longer detects gestures from the locked mouse. In that case, I think GCMouse will be the way to go.
On the other hand, I’d definitely suggest not locking the mouse by default as if the user wanted to draw a cursor, the system cursor is much more responsive as it isn’t tied to the app’s frame rate.
As for the suggested mouse API, it looks like it could work well, though we’d also need mouseConnected & mouseDisconnected callbacks for obvious reasons. Possibly even calling these (if implemented by the user) for each mouse connected at app startup rather than have the user poll for availability directly.
The mouse:release() function should be fine though (although you won’t be able to do this & capture per mouse) as you can just change the return value in prefersPointerLocked and call setNeedsUpdateOfPrefersPointerLocked
I’m glad to hear keyboard support is on the way though, can’t wait to give it a go.
Thanks!
That all sounds fine, would be cool to play around with multiple mice too
Some progress on this particular topic: https://twitter.com/johntwolives/status/1364506056559267841?s=20
Looking good can’t wait to get my hands on it!
Judging from the comment I assume there’s only the polling API currently?
@John - very impressive, looking forward to using the API for both keyboard and joypad. Note you have both the iPad and joypad wired up - are they both for power? I was assuming the API would be bluetooth/wireless based?
Also - will there be a broad range of peripherals that can use this, I assume there may have to be a calibration module to enable different kit to be identified and responses defined?
@Bri_G - The API provided by Apple is independent of the connection and I’m not even sure they support a wired connection for that matter. It looks like the controller isn’t connected to the iPad so I’d assume just power.
The app receives a list of the connected gamepads and provides access to them. As for additional peripherals, Any device listed here should work in the same way without any additional work from John: https://support.apple.com/en-gb/HT210414
@Bri_G @Steppers In that video I just had the DS4 connected to a power brick because it kept running out of juice. I don’t have an adapter from MicroUSB to USB-C so not sure if that works on iPad. It works wired with my Mac though
When I was testing I managed to freeze all input to both my Mac and iPad in the exact same way so I can only assume they use the same code. Even the haptic feedback on my trackpad stopped o_o
By the looks of it the DS4, Xbox controller work and others probably do too. It depends on Apple’s support for them. The PS5 controller is supported in 14.5 beta as well
I currently only have polling working but I’ll also be able to add all the callback handlers too as well as the connected/disconnected notification handlers
@Steppers - thanks for the link, looks faesible.
@John - thought it was just power. May need to watch out for clashes on my Mac and iPad . Thanks again.
Is this going to be released soon?
All,
Just been searching on the net for keyboards, mice and joypad that you can use with an iPad. Any suggestions from here?
Also - @John clarification of the API you are interfacing. Does the API include commands to identify inputs for joypad buttons etc. There are a few joypads etc from different manufacturers - is there one consistent interface or do you have to specify say a keyboard key alternative. Will the API include calibration of buttons etc. Also does it operate in different modes -proportional/analog. Finally some systems include both bluetooth and wifi interfacing.
Early days I know but just need to understand what I should be considering for an input device.
Hi,
I believe any usb keyboard or mouse should work though I assume you’d prefer bluetooth ones. For those, I would personally try to grab a keyboard from a reputable brand as the keyboards on the really cheap ones are truly dreadful in my experience. I bit the bullet and grabbed a Logitech folio touch keyboard case (with touchpad) which has been fantastic if a little pricey.
As for gamepads, I’ve never had a great experience with other brands so tend to stick with either an Xbox or PS4 controller. Some other controllers are often missing the L3 & R3 buttons in the joysticks too which can be a pain at times.
@Steppers - thanks for the feedback. Do you need a usb adapter for the iPad connector to use usb joypads? I have a small bluetooth keyboard, which I have used but looking at Logitech ones at moment. There’s one which interfaces to iOS, and Android and Windows at the flick of a switch - thinking of getting one.
I tried using PS4 pads on my Mac but couldn’t get it to work - apparently needs OS after Mojave, but Mac runs too slow and not as compatible with later OSX. Also need iOS 14 on the iPad. That’s the fruit for you. I have 14 on my pad so will try the PS4 pads on my iPad. Thanks again.
Ah most likely yes. I’ve got a USB-C iPad and I use a C to Full-size A adapter when I do USB stuff. No idea about usb with the lightning connector but my first response would be it’s not possible.
I think I’ve seen those ones before, don’t look half bad. Just check the reviews obviously.
If you have iOS 14 then it should just work thankfully.