Enabling Cooking Experiences

How can existing kitchens be made accessible to people with visual impairments and complete blindness in such a way that dangerous situations can be prevented, and help can be provided with cooking.

Throughout the process we conducted interviews kept in touch with our participants to verify our decisions and eventually iterate on them.

Charly is an intelligent kitchen aid that can be integrated into all kitchens. The approach is based on the principles of a learning artificial intelligence as well as voice input and output. A mobile camera enables local, precise help anywhere in the kitchen.

We analysed our design process after we finished the project. It shows that we went through an iterative process. We were able to incorporate the learnings from the previous steps into the further decision-making processes so that we were able to create a detailed result.

For this project we have used various research methods that should guide our decision-making process.

Several user interviews helped us to find out what our users think about our design solutions and what different problems they encounter in their everyday life. In addition, we wanted to prove the relevance of our project and gain detailed insights.

While Michaela's greatest challenge is orientation outside, Vanessa and Nadira see more need for support in the kitchen. That is where the greater everyday limitation lies.

A lot of freedom and independence can be offered if blind people can prepare their own lunch.

Vanessa needs support and relief for her eyes so that she has to resort to the rest of her vision less often. Michaela and Nadira instead rely on full support and guidance.
In addition, a warning function can prevent all dangers that affect everyone equally.

People with severe visual impairment are particularly dependent on their hearing and the haptics when using devices. That is why, for example, they use clocks, scales and measuring tapes with voice output. In addition, preference is given to using products whose shape and feel allow conclusions to be drawn about their operation. Touch surfaces, however, can hardly be used.

With the help of markings and an orderly household, the blind and visually impaired can orient themselves in their kitchen. The use of smartphones or laptops is possible via screen readers, but it does take some time getting used to.

The quality of the food can be improved through smart sensors and the dangers of burning and boiling over are prevented. Voice assistants can control smart home devices via a voice interface and provide information from the Internet.

Another promising technology is image processing, which enables text and products to be recognized. In a test with the Google Lens app, it was able to distinguish a peanut kernel from a roasted corn kernel.

Due to Corona, it is not possible to observe the interviewees cooking independently and to analyse their behaviour. We tested cooking blind ourselves. This method of self-experimentation is not representative, but helps in developing the project to assess problems and necessary assistance.

The labeling and closer inspection reveal important interactions in the kitchen. This includes, for example, the rotary controls of the oven, the touch control of the hob and the entire structure of the kitchen.

The Value Proposition Canvas (VPC) is a method to examine the value proposition for the target group and to design the product accordingly. It analyzes which tasks the target group wants to do and which positive (gains) and negative experiences (pains) can arise.

The blind and visually impaired want to set appliances in the kitchen independently and use kitchen utensils to be able to prepare food. The gains include, among other things, having fun with the cooking process, the choice of dish and that it tastes good. Injuries and inedible food, on the other hand, are pains.

In addition, the device has a personality and guides you through the individual steps of recipes. The cooking assistance system keeps an overview of the kitchen, shows the current status and warns of dangers in good time.

At the beginning of the project, we worked with the aim of creating an orientation aid for people with visual impairments and the blind. However, the interviews revealed that a much bigger problem lies elsewhere.

With the "Crazy 8" method, we were able to collect our first ideas and put them down in sketches. Although we dealt with a different problem in the further course of the project, we were still able to use the ideas. In particular, the technologies and communication between technology and people.

Statistically speaking, most of the accidents to which blind people are more prone happen in the home. There is a greater need and a greater gap in the market in this area. All three interviewees confirmed that such help and support can be relieved by a linguistic input and output system.

Based on this knowledge and the specific problem, it helped us to formulate a product vision. This enabled us to make better decisions together as a team, as we now knew what we were working towards.

“For the visually impaired and blind people who cannot or are not allowed to cook independently or on their own, Charly is a kitchen helper who is equipped with cameras and voice technology to help to enable independent and safe cooking in your own kitchen. "

We thought very broadly how we can help our target group and generated quantitative ideas.

The round shape of the prototype can be explained by the fact that the assembly of the modular system should be simplified as much as possible. The target group can use the modular system without having to pay attention to alignment or accuracy of fit.

In addition, the outer edge of the camera head is slightly curved inwards. The operation is ergonomically adapted and also makes it easier to feel the camera lenses so as not to cover them up.

Due to its strong expression, the color orange is a signal color and can be better perceived by people with the least amount of vision. White also functions in a similar way. The light color reflects the available light most strongly, so that the visual recognition of the device is possible for people with the least amount of vision.

To test this idea, we analyzed a use case with a workflow of a three-course menu. As with the self-experiment, care is taken to cover the largest possible range of work steps in the kitchen.

These individual work steps are analyzed to determine which speech input and output is required in order to be able to provide the correct assistance so that dangerous situations can be prevented.

During this analysis it is noticeable that all of Charly's auditory functions can be divided into the categories of confirmations, notices and warnings.

After testing and defining solutions we came up with a feature list which opose a solution to our given challange.

Through the linguistic interaction with the product, the blind or partially sighted can make all settings themselves and does not need a sighted accompaniment.

The humanity of the voice output and the interaction with the artificial intelligence enables a relationship of trust.

This is necessary because the blind or visually impaired must completely rely on Charly. Especially since such a relationship between man and machine and their full support makes people less prudent and more careless.

In conclusion, it can be summarized that despite the circumstances and difficulties that have developed in the course of the project, a concept was developed that can help blind and partially sighted people cook in their own kitchen.

By implementing this concept, the number of dangerous situations could be drastically reduced. In addition, the self-confidence of those affected would be strengthened in the long term, so that they can cook independently and without fear with the help of Charly.

In order to evaluate the existing concept, user tests can be carried out with the target group in the next step. These would form the basis for adapting and adding functions. After the positive evaluation of the concept, the scope of a minimal viable product could be determined and the costs required for it could be determined.

This is probably the most important thing that we have learned. After the point in time in the design process had progressed significantly and went beyond the conception, the needs of those affected were decisive for the switch to a new concept.

The knowledge and the findings from the first research and conception could also be applied to the new concept in many ways. So it became clear: The first approach and concept development are not in vain if the focus is changed. A lot of valuable things can be learned for the new concept and for future projects.

Especially after the late change in the concept, this was essential for working in a team. Tasks were distributed and each group member had their own area of responsibility. With a carefully managed schedule, the progress of the project can be kept in view and it is possible to estimate at an early stage when the time could be running out.

This makes it possible to point out a delay and to add comments on what has been worked out. In this way, errors can be corrected at an early stage.