Enabling Cooking Experiences
At the end of 2019, around 580 thousand people with visual impairment and blindness were living in Germany.
I was part of a project that aimed to develop a concept of giving blind people back their ability to cook on their own.
How can existing kitchens be made accessible to people with visual impairments and complete blindness in such a way that dangerous situations can be prevented, and help can be provided with cooking.
Throughout the process we conducted interviews kept in touch with our participants to verify our decisions and eventually iterate on them.
Charly is an intelligent kitchen aid that can be integrated into all kitchens. The approach is based on the principles of a learning artificial intelligence as well as voice input and output. A mobile camera enables local, precise help anywhere in the kitchen.
December 2020 - January 2021 (5 weeks)
Research, UX, UI, Prototyping
The corona pandemic was still ongoing and because of new restriction by the government, we were restricted in meeting more than one person. Planned shadowings and user testings could not be carried out.
We analysed our design process after we finished the project. It shows that we went through an iterative process. We were able to incorporate the learnings from the previous steps into the further decision-making processes so that we were able to create a detailed result.
For this project we have used various research methods that should guide our decision-making process.
Several user interviews helped us to find out what our users think about our design solutions and what different problems they encounter in their everyday life. In addition, we wanted to prove the relevance of our project and gain detailed insights.
"I burned my hair once while cooking, since then I've been afraid to go to the stove alone. "
"I can't find my way outside at all on my own, I always need company."
"Thanks to the mobility training, I know some routes by heart and so I can manage on my own."
“I can't cook alone. We are also very limited in the living group and are not allowed to do a lot alone. "
"Outside I rely a lot on my hearing, I don't have any major problems there."
“Cooking is quite normal for us. I mostly cook, but I also like to do it. "
While Michaela's greatest challenge is orientation outside, Vanessa and Nadira see more need for support in the kitchen. That is where the greater everyday limitation lies.
A lot of freedom and independence can be offered if blind people can prepare their own lunch.
Vanessa needs support and relief for her eyes so that she has to resort to the rest of her vision less often. Michaela and Nadira instead rely on full support and guidance.
In addition, a warning function can prevent all dangers that affect everyone equally.
People with severe visual impairment are particularly dependent on their hearing and the haptics when using devices. That is why, for example, they use clocks, scales and measuring tapes with voice output. In addition, preference is given to using products whose shape and feel allow conclusions to be drawn about their operation. Touch surfaces, however, can hardly be used.
With the help of markings and an orderly household, the blind and visually impaired can orient themselves in their kitchen. The use of smartphones or laptops is possible via screen readers, but it does take some time getting used to.
The quality of the food can be improved through smart sensors and the dangers of burning and boiling over are prevented. Voice assistants can control smart home devices via a voice interface and provide information from the Internet.
Another promising technology is image processing, which enables text and products to be recognized. In a test with the Google Lens app, it was able to distinguish a peanut kernel from a roasted corn kernel.
Due to Corona, it is not possible to observe the interviewees cooking independently and to analyse their behaviour. We tested cooking blind ourselves. This method of self-experimentation is not representative, but helps in developing the project to assess problems and necessary assistance.
The labeling and closer inspection reveal important interactions in the kitchen. This includes, for example, the rotary controls of the oven, the touch control of the hob and the entire structure of the kitchen.
Value Proposition Canvas
The Value Proposition Canvas (VPC) is a method to examine the value proposition for the target group and to design the product accordingly. It analyzes which tasks the target group wants to do and which positive (gains) and negative experiences (pains) can arise.
The blind and visually impaired want to set appliances in the kitchen independently and use kitchen utensils to be able to prepare food. The gains include, among other things, having fun with the cooking process, the choice of dish and that it tastes good. Injuries and inedible food, on the other hand, are pains.
In addition, the device has a personality and guides you through the individual steps of recipes. The cooking assistance system keeps an overview of the kitchen, shows the current status and warns of dangers in good time.
At the beginning of the project, we worked with the aim of creating an orientation aid for people with visual impairments and the blind. However, the interviews revealed that a much bigger problem lies elsewhere.
With the "Crazy 8" method, we were able to collect our first ideas and put them down in sketches. Although we dealt with a different problem in the further course of the project, we were still able to use the ideas. In particular, the technologies and communication between technology and people.
Statistically speaking, most of the accidents to which blind people are more prone happen in the home. There is a greater need and a greater gap in the market in this area. All three interviewees confirmed that such help and support can be relieved by a linguistic input and output system.
“How can existing kitchens be made accessible to people with visual impairments and complete blindness in such a way that dangerous situations can be prevented, and help can be provided with cooking.”
Based on this knowledge and the specific problem, it helped us to formulate a product vision. This enabled us to make better decisions together as a team, as we now knew what we were working towards.
“For the visually impaired and blind people who cannot or are not allowed to cook independently or on their own, Charly is a kitchen helper who is equipped with cameras and voice technology to help to enable independent and safe cooking in your own kitchen. "
We thought very broadly how we can help our target group and generated quantitative ideas.
The first thought deals with small gadgets that communicate through a main device. This idea is very expensive because a lot of technical equipment would have to be bought.
Another idea is a Thermomix for the blind. This is discarded because it does not enable the cooking experience, but creates its own form of cooking. In addition, no all-round view is possible to point out any dangerous situations.
The idea of the final concept is a device that is placed in the kitchen and can be added modularly if necessary. This allows the entire kitchen to be kept in view and provides protection against dangerous situations.
The round shape of the prototype can be explained by the fact that the assembly of the modular system should be simplified as much as possible. The target group can use the modular system without having to pay attention to alignment or accuracy of fit.
In addition, the outer edge of the camera head is slightly curved inwards. The operation is ergonomically adapted and also makes it easier to feel the camera lenses so as not to cover them up.
Due to its strong expression, the color orange is a signal color and can be better perceived by people with the least amount of vision. White also functions in a similar way. The light color reflects the available light most strongly, so that the visual recognition of the device is possible for people with the least amount of vision.
To test this idea, we analyzed a use case with a workflow of a three-course menu. As with the self-experiment, care is taken to cover the largest possible range of work steps in the kitchen.
These individual work steps are analyzed to determine which speech input and output is required in order to be able to provide the correct assistance so that dangerous situations can be prevented.
During this analysis it is noticeable that all of Charly's auditory functions can be divided into the categories of confirmations, notices and warnings.
After testing and defining solutions we came up with a feature list which opose a solution to our given challange.
These are used to give Charly an all-round view. The overlapping perspectives give the AI a better three-dimensional idea and can guarantee the best possible assistance even with hidden cameras.
Add-on camera heads
The camera heads are designed so that any number can be added. This means that larger kitchens can also be equipped. For charging, the heads can simply be stacked on top of each other on the speaker. If necessary, your own smartphone can also be used as a camera.
Charly and his camera heads have an internal battery. The product can therefore be used on the move. The power connection is located in the fastening base of the device so that the cable does not have to be plugged in and unplugged for every mobile application. The charging contacts thereby simplify the charging process. The magnetic connection, together with the indentation and notch, ensures that the camera heads are held precisely.
Charly communicates with the help of the built-in microphones and the 360 ° loudspeaker. In this way, optimal audibility is guaranteed despite any possible orientation of the device.
The voice can be configured not only in terms of volume, but also in terms of gender. This enables a more individual relationship and perfect comprehensibility. That is why the product bears the unisex name: Charly.
Through the linguistic interaction with the product, the blind or partially sighted can make all settings themselves and does not need a sighted accompaniment.
The humanity of the voice output and the interaction with the artificial intelligence enables a relationship of trust.
This is necessary because the blind or visually impaired must completely rely on Charly. Especially since such a relationship between man and machine and their full support makes people less prudent and more careless.
Results & Next Steps
Summing it up
In conclusion, it can be summarized that despite the circumstances and difficulties that have developed in the course of the project, a concept was developed that can help blind and partially sighted people cook in their own kitchen.
By implementing this concept, the number of dangerous situations could be drastically reduced. In addition, the self-confidence of those affected would be strengthened in the long term, so that they can cook independently and without fear with the help of Charly.
In order to evaluate the existing concept, user tests can be carried out with the target group in the next step. These would form the basis for adapting and adding functions. After the positive evaluation of the concept, the scope of a minimal viable product could be determined and the costs required for it could be determined.
"It is perfectly fine to change the focus in the middle of the process or even to throw the concept overboard when it only becomes clear that there is more urgency or need elsewhere."
This is probably the most important thing that we have learned. After the point in time in the design process had progressed significantly and went beyond the conception, the needs of those affected were decisive for the switch to a new concept.
The knowledge and the findings from the first research and conception could also be applied to the new concept in many ways. So it became clear: The first approach and concept development are not in vain if the focus is changed. A lot of valuable things can be learned for the new concept and for future projects.
"A detailed schedule and the exact structuring of the process is helpful in order to complete the project at the end without major time problems."
Especially after the late change in the concept, this was essential for working in a team. Tasks were distributed and each group member had their own area of responsibility. With a carefully managed schedule, the progress of the project can be kept in view and it is possible to estimate at an early stage when the time could be running out.
"Regular updates from the other team members give everyone an overview of the status of the others and the general status of the process."
This makes it possible to point out a delay and to add comments on what has been worked out. In this way, errors can be corrected at an early stage.