“India is so dynamic, I love going there,” remarked Apple CEO Tim Cook, a day ahead of the tech giant’s annual Worldwide Developer Conference (WWDC) keynote, where the artificial intelligence strategy as well as the next versions of software including iOS, iPadOS and macOS will be revealed. Cook, alongside Apple’s CEO-elect John Ternus, met distinguished winners of the Swift Student Challenge. This underlines Apple’s continued focus on the developer community, and gives student devs a chance to showcase their creativity and coding skills.
Distinguished winners who presented their creations to Cook and Ternus include India’s Gayatri Goundadkar with the Steady Hands app, Germany’s Anton Baranov who developed a pitch coach tool leveraging Apple’s Foundation Models framework, South Korean Yoonjae Joung who has used Anthropic’s Claude, OpenAI’s Codex and Google’s Gemini to build the LeViola musical instrument learning app using Apple’s CoreML, and Rino Ito from Japan who developed the ColortheGoalDaruma app which focuses on goal-setting inspired by a traditional Daruma doll.
“Gayatri’s app is a beautiful example of how technology can empower creativity and make art more accessible for everyone. By harnessing the power of iPad and Apple Pencil, she is helping individuals with tremors express themselves and share their art with the world. It’s so inspiring to see Indian developers like Gayatri use their talents in service of others, and I know her future is bright,” said Apple CEO Tim Cook.
Also Read:A new silicon equation, and the great PC reset
The underlying theme for this year’s winning submissions from 37 countries is the use of artificial intelligence (AI) for accessibility use cases.
Goundadkar, a 20-year-old third-year computer science student at Maharashtra Institute of Technology World Peace University, tells HT in a conversation that the idea for the Steady Hands app came about one day when her grandmother was diagnosed with essential hand tremors (or Essential Tremors, or ET). This is a neurological disorder that causes involuntary, rhythmic shaking, most commonly in the hands, and she couldn’t make art any more.
“As a child, I loved creating things. I used to sit hours sketching with my grand mother with a sketchbook and a pencil. After a lot of research, I found out that my grandmother isn’t alone,” she explains.
Steady Hands, an app which Goundadkar says is awaiting patents and will soon be listed on the Apple App Store, wants to create a solution for millions of people worldwide who are suffering from essential tremors as a condition. The app, she says, was built and ready in just about a month’s time.
As for the existing solutions available, she notes that “they only focus on helping a person eat or track medication or track surgery, but there was nothing out there which could help a person draw or write again.” Steady Hands uses AI to calculate and compensate for tremors when fixing a drawing or a sketch or a handwritten element created using an Apple Pencil, to look as it was intended to by a user.
Goundadkar’s Steady Hands app uses Apple’s developer focused frameworks including SceneKit and RealityKit and created using Swift. At the core of the app is the Accelerate framework on which the basis of the app is built—that is, an ability to take a test to analyse raw motion data to identify and determine a user’s tremor intensity and frequency. This creates, what Goundadkar calls, a personalised tremor profile.
While multiple users cannot set up their accounts on this app on the same iPad for instance, she does confirm that this test can be taken multiple times and will tweak the app’s features accordingly.
“When someone starts drawing, it will automatically detect the tremor in their drawings to remove those tremor movements and keep the intentions,” she explains. Goundadkar says much of her research was a mix of medical and behavioural. “Tremor isn’t the same for every individual and I had to understand the medical reasons. Severity, direction and consistency differs, which is why I simply can’t use a number based system,” she says.
Apart from Apple’s developer focused tools, Goundadkar says Anthropic’s Claude and OpenAI’s ChatGPT helped her a lot with research. “They also were very useful when I had to integrate RealityKit and SceneKit because I had a very short span of time,” she explains. Steady Hands was ready for submission as part of Apple’s Swift Students Challenge for this year, in the span of a month.
Also Read: Tech Tonic: Reassessing the subscription bundle
For someone so young, clarity isn’t Goundadkar’s weak link. She is very clear on the vision for the app, and admits the rough edges. “I could have made the interface much better and that’s something I am working on. There are a lot more accessibility features too,” she says.
“Hand tremor can cause someone to lose their entire identity. It is not only about losing passion. During testing, I came across people who were not able to do basic tasks such as signing cheques,” she points out. When asked about a possible feature that helps users beyond creating art into more every day tasks such as document signatures, Goundadkar points out that she had developed a feature called Signature within Steady Hands, with the idea being to help users with the task. But that iteration, she says, proved complex for users.
“What I focussed more on was adding different types of exercises which help with all kind of hand movements, so they are able to have control over them with time. These exercises will help them to eat, will help them to do their signature, and will help them to write again,” she says. Another thing young Goundadkar refuses to compromise on, is an intent to receive feedback from the user community. As that base widens, it will be key to understand usage trends and integrate features that are requested for, in future iterations.
