Drawxi: An accessibility tool for team collaboration

PROJECT OVERVIEW

Visual impairment can profoundly impact social well-being and professional advancement. In the modern workplace where all manner of visual communication is employed, the visually impaired are left at a disadvantage in communication and consequently, collaboration. Drawxi is an audio-haptic enabled tool that enables two-way communication and collaboration between sighted users and those with those visual impairments.

ACHIEVEMENTS

Finalist in CHI 2019, Glasgow (Conference on Human Factors in Computing Systems)
Finalist in the Cheng Wu Innovation Challenge 2019

TEAM MEMBERS

Suraj Chiplunkar
Anany Maini
Dinesh Ram
Zixuan Zheng
Yaxin Zheng

ROLE IN PROJECT

UX researcher
UX designer
Video director

METHODS AND SKILLS

User interviews
User journey mapping
Stakeholder mapping
Contextual inquiry
Rapid prototyping
Co-designing
Videography

The challenge

Current solutions fail to provide an affordable, user-friendly collaborative platform for visual communication between visually impaired and sighted people. This is further exacerbated by the march of technological development which continues to predominantly explore the realm of visual communication. This issue can manifest itself in the following ways— difficulty synchronizing team understanding of concepts, lower engagement in the decision making processes and implicitly, lower expectations of team members with visual impairments.

CASE 1:

Sighted collaborator communicating with a visually impaired colleague.

CASE 2:

Visually-impaired person communicating with a sighted colleague.

If the roles are reversed, the visually impaired individual has no pragmatic way to produce visual representations. This of course means that their sighted colleague is deprived of any visual input. Sight being the dominant sense of sighted individu… — If the roles are reversed, the visually impaired individual has no pragmatic way to produce visual representations. This of course means that their sighted colleague is deprived of any visual input. Sight being the dominant sense of sighted individuals, the conversation is encumbered.

Sighted users rely on sight and sound for their intake of information whereas users with visual impairments rely first on sound and then through tactile means. This fundamental incongruence in the preferred channels of communication necessitates a mediator within the system which can facilitate communication, much like the role played by a translator in a conversation between individuals who speak in different tongues.

Furthermore, it was realized that due to this barrier in communication, the generative capabilities of visually impaired individuals is largely lost, thus this design presents value in not only gaining inclusivity but allowing these lost ideas to see the light of day.

Solution Overview

Drawxi is a system comprised of two components— the audio-haptic Drawxi board and a web-based Drawxi Canvas. Collectively, these two components make for a seamless experience of communication between sighted and visually impaired parties.

DRAWXI CANVAS

The Drawxi board is designed for use by visually impaired users as an input device as well as an output device. Visual information produced by sighted users can be perceived through the haptic actuators in a grid-like arrangement. The figures are further described through labels which are received in an audio form. Output of shapes is also achieved through tactile means by the visually impaired users.

DRAWXI CANVAS

For production of figures in a format compatible with the Drawxi board, sighted users may use the Drawxi canvas to produce the figures using a drawing toolset that is identical to that of the Drawxi board such that no information is lost in translation.

How it works

This video gives a run-down on how these two components come together to perform as one system and how it seamlessly integrates in a setting that consists of sighted and non-sighted individuals.

Research process and methodologies

In pursuit of a solution, it was immediately obvious to us that we would have to use research methods that were inclusive and the design process was to be driven by our target group.

A fundamental challenge in designing accessible products is ultimately to do with the (lack of) capacity to adequately fill the shoes of someone who is differently abled.

INTERVIEW STUDIES

Interview studies served to confirm initial assumptions regarding pain points in visual communication. Various workarounds were described where participants broke down complex ideas into a list and verbally described each point sequentially, using an Excel sheet, Word file, or mobile device. Ad hoc representation of visual concepts require tactile material which is seldom available in immediacy. Consequently, visually impaired participants seldom shared their feedback or critique for graphical diagrams.

INSIGHTS FROM INTERVIEWS

Individuals felt left out of conversations where visuals were used as supplementary material.
They were also hesitant to ask for clarification on concepts lest they appear burdensome.
Production of visual information was not practical since they preferred production through tactile means and materials for such production was not readily available at hand.
Visual information, once produced, was not retrievable in any accessible format.
An additional burden was being placed upon sighted users who had to correctly describe graphical information verbally as well as interpret the conceptual knowledge being conveyed by their visually impaired colleagues.

CONTEXTUAL INQUIRIES

Two contextual inquiry sessions were performed to study the communication techniques employed by visually impaired individuals in a mixed-population environment. We also intended to study the workarounds they utilized to overcome inaccessible graphical information. Observations revealed that participants had varying strategies to explain complex ideas. One participant used a sequential stepwise description based on memory which did not need graphical information, while the other participant utilized lived experiences to explain ideas and used primitive stick figures as illustrations. These illustrations would however, only be comprehensible if accompanied by live explanation or a textual description.

Inquiries were conducted in public as well as private settings. Observations were focussed on their interactions with their surroundings, artifacts and their manners of communication with other individuals.

MENTAL MODEL AND TASK ANALYSIS

As a direct result of the contextual inquiries in public settings, we constructed a flowchart of the decision making processes in everyday life. In this instance, the scenario that was chosen for analysis was grocery shopping at a grocery store that was familiar to the participant. Notes were compared between the way we conducted our own tasks and how the same tasks were handled by a visually-impaired individual.

Task analysis solely for visual representation was conducted in private settings where needs were discussed and their current workarounds that this situation gave rise to. The work arounds included description in a list format, spreadsheets, etc.

The tasks marked with blue circles were noted as tasks that were unique to persons with visual impairments. The entire flow chart can be viewed here.

INSIGHTS FROM CONTEXTUAL INQUIRIES

Basic drawing (as well as writing) is achieved through the use of both hands, with one finger denoting a reference point based on which all other shapes are placed.
Objects were perceived by tracing the contours first, and then exploring physical features inwards.
Congenitally blind individuals visualize things differently compared to sighted individuals. Most visualizations consist of objects in their profile view or most defining view.
Closing shapes while drawing them was found to be a particularly challenging task necessitating some form of technological assistance.

CO-DESIGN WORKSHOPS

We performed two Co-Design workshops with ten visually impaired participants. One of these sessions was conducted at the Joint Indiana and Ohio convention by the American Council for the Blind. These workshops focussed on specific task-centered activities. After a brief introduction, "What-if" cards helped participants kickstart brainstorming. To enable rapid prototyping, we used materials like clay, popsicle sticks, stickers, stencils, and tactile UI elements (created by laser cutting soft cardboard).

Workshops were conducted with participants ranging from their mid-twenties to senior citizens. Different degrees of visual impairment of the participants was also a consideration.

For the purpose of building prototypes, tactile materials were used. Participants took the initiative to explore the available materials and familiarize themselves with their forms.

Once the designs had been iterated upon, the pieces were taped to a larger board and handed to other participants so they might explore the prototype. Further, presentations of the designs were given by the participants themselves without our involv… — Once the designs had been iterated upon, the pieces were taped to a larger board and handed to other participants so they might explore the prototype. Further, presentations of the designs were given by the participants themselves without our involvement so they might justify their own decisions.

INSIGHTS FROM CODESIGN WORKSHOPS

Tracing shapes using fingers proved to be more accurate than drawing through the use of a stylus.
Congenitally blind individuals visualize things differently compared to sighted individuals. Most visualizations consist of objects in their profile view or most defining view.
Closing shapes was found to be a particularly challenging task necessitating some form of technological assistance.
Elderly individuals with reduced sensitivity in their fingertips might be impeded in their tactile sensing of shapes.

Design iterations and evaluation

Testing was performed with iterations of the physical form as well as the context of use. The navigation and shape buttons proved to be comfortable for use by both the left or the right hand. Participants found some difficulty in identifying the shapes by the size of the buttons so that is something that would need to be further iterated upon.

PHYSICAL FORM EXPLORATIONS

We went through many iterations with the surface texture as well as the dimensions of the board. We tried smooth surfaces, matte surfaces, dotted protrusions and line grids for the active drawing area. In the end, a matte surface proved to be conducive to the drawing experience. The dimensions of the active area were restricted to the dimensions of the hand since that allowed the participants to gauge distances and proportions better with their non-dominant hand as a reference.

Primary considerations

Material of the board and drawing surface.
Position of use (On top of desk vs on the lap).
Dimensions of the drawing surface.

Early stage concept sketch of the device. Initial designs tended to be divided into four quadrants with a cross in the middle to use as a reference point. This idea was supplanted with a virtual grid system and the device dimensions were made smalle… — Early stage concept sketch of the device. Initial designs tended to be divided into four quadrants with a cross in the middle to use as a reference point. This idea was supplanted with a virtual grid system and the device dimensions were made smaller.

Different materials and textures were explored. Some tended to obstruct the drawing process and distract from free expression. A matte and even surface won the final verdict.

DRAWING EXPERIENCE

Finger drawing was vastly preferred over stylus drawing as it was a more direct touch point on the active area. It also served to more accurately judge distances between points on the surface. Each figure could be drawn with a double tap of the finger and the starting point would remain active in haptic feedback so it would not get lost in the process. Since some basic shapes are drawn more than others, shorthand notations would allow the user to draw them quickly by using the quick-draw buttons on top.

Primary considerations

Finger drawing vs style use.
Use of forms and shapes
Hand postures

Storytelling and narrative forms were explored here, where the participant drew simple human figures and narrated a fictitious story.

After vigorous debate about stylus drawing versus finger drawing, we went in favor of finger drawing since spatial perception seemed stronger with finger drawing. Further, there was the issue of misplacing the stylus.

INFORMATION RETRIEVAL

Based on our insights, we chose haptic actuators with which figures could be traced back. Moreover, since the active area was restrictive in size, the complete canvas would be accessible by navigating a grid-like arrangement of instances. This was in line with the mental models which were revealed through the contextual inquiries and co-design workshops.

Primary considerations

Raised forms vs haptic feedback
Role of sound in information retrieval
Current retrieval patterns

Considerations were made for how the system would help in identifying the shapes that have been drawn on the board or on the Canvas interface. Audio output was minimized so as to not interrupt smooth communication between team members.

We also took inspiration from the way they interacted with their devices everyday. Familiarity and intuitive interaction was one of the foremost concerns with this design. — We also took inspiration from the way they interacted with their devices everyday. **Familiarity and intuitive** interaction was one of the foremost concerns with this design.

Proposed workflow with Drawxi

Drawxi can be best explained through a user scenario of its use in a fictitious meeting environment. Further, Drawxi also allows users to collaborate remotely through the same workflow depicted above.

1) A visually impaired user wishes to propose an alternative organizational structure to improve transparency.

2) Begins engagement by establishing the position and gauging the size of the active area.

3) Drawing begins with a double tap.

4) Haptic feedback follows the finger while the starting point remains active. This constant feedback lets the user know that drawing is in progress.

5) To add more elements the user uses the navigation buttons and shifts to the next frame on Drawxi Canvas.

6) While the user explains the new organizational structure, the depiction is visible for people to see on Drawxi canvas.

7) A co-worker suggests a minor addition by adding new shapes to a new frame on Drawxi Canvas.

8) The user gets a notification on Drawxi board and the user presses the OK button. The user is taken to the frame where the minor addition is added.

Potential challenges to the product

SOCIAL ACCEPTANCE

Perhaps the first test Drawxi will face is not technical, but cultural. Drawxi is a tool for inclusivity and as a tool, unless social barriers in the workplace are overcome, there is little hope for adoption. There needs to be a change in the current work environment where everyone is considered to be a value generator and an equal contributor to the ideas that make future possible.

BUDGETARY CHALLENGES

Once Drawxi has been introduced to the market, we will get a good estimate of the it will be required to fine tune the balance between features and price of the product which is admissible to the users. While this initial iteration is fine-tuned to the best of our abilities, the market journey of the product will no doubt reveal several things which have been overlooked at this nascent stage.

A GENTLE LEARNING CURVE

As with any new technology, user on-boarding has the power to make or break a product experience. A tutorial or an accessible manual has to be designed meticulously so that it not only gives a good idea of how the product works but also an idea of the potential it has of fitting into their everyday life as a phone does. We suspect that a large portion of our energy will be dedicated to the design of an ideal onboarding flow before release.

In retrospect

Walk the talk

Interviews were easiest to conduct but no form of research was more fruitful than that which involved hands-on work and allowed participants to try out a tangible prototype.

Flow with the process

Things don’t always turn out the way you would expect. This entire project began as an exploration in making payment systems accessible but we pivoted in a direction that tackled a more fundamental problem.

A bird in the hand is worth two in the head

No design is finished. There are always kinks and creases that need to be ironed out, but making an imperfect something and then making it better is always preferable to having nothing at all.