UX Studies 1 - Lessons learned from the Feedback Wall (that don't have much to do with feedback)

The Feedback Wall at my library has been ongoing since mid-February and was inspired by this post on the UKAnthroLib blog. I'll admit that my secret hope when putting it together was that we would get positive feedback in among the mixture of suggestions and criticisms, the way other libraries seemed to be getting. In hindsight, I know that was a lot to ask given the relationship we currently have with our students, and our feedback so far has been entirely criticism and suggestions. While it has occasionally been a bit wounding to the ego, a lot of good has come out of the Feedback Wall and we intend to keep it going. Hopefully we'll get to the point where we do start to get positive reinforcement from students, but I certainly know now not to expect it.

Along those lines, I wanted to share a few lessons I have learned during this process that have less to do with the comments people have left and more with the process of soliciting feedback in this way and about workplace culture.

1- You can provide all the bells and whistles you want as long as you also address the most pressing needs.

I had been feeling pretty good about the Feedback Wall. In response to feedback we've received in the last three months we've brought in pens for next to the catalogue computers, sped up the acquisitions process for recommended books, purchased USB DVD drives and book rests for students to borrow, acquired ear plugs to give out to students and relaxed our drinks rules to allow students to finish hot drinks in the Issue Desk area rather than outside the library.

However, one issue keeps coming up consistently: the students want a water cooler or drinking fountain in the library. It is our most common suggestion/complaint on the feedback wall and was one of the most common comments on our user survey last year. No matter what else we were doing, this issue came up again and again and, while I sympathise very much with the student perspective, there was reluctance from higher up to do anything about it. I feel certain it will keep coming up until we do take some kind of action. It has been an excellent lesson in the fact that even if you think your institution is being reasonable (i.e. bring in your own water bottler), the users do not necessarily agree, and it is important to address those concerns.

2 - No matter how hard you work, people will still find something to critique you on. There's a balance to strike between trying to meet every single need and not bothering.

As a library assistant I work very hard to make the library a positive place for students to work. I am one of their main human points of contact with the library and as such I am very invested in making sure they have a good experience. However, I am also in the position of enforcing rules and, as the person who keeps track of the Feedback Wall, representing the views and expectations of my institution to students. This is a very difficult position to negotiate, especially when your sympathies and the policies you are enforcing do not align.

Even if you could somehow meet every need thanks to an inexhaustible budget, incredibly flexible library space and a staff entirely comprised of the sort of service-minded, bend-over-backwards librarians we wish we could be, there will always be something that someone thinks you've done wrong or could improve on. If you're in a more flawed (i.e. real) library, there are lots of somethings users will pick up on. This can become overwhelming if you let it. Or, you can take things on as you have the capacity to deal with them, one step at a time. Like I talked about in my previous post, keeping your focus on the final goal of a perfect library is unattainable and will ultimately lead you to burn out. Do what you can, when you can. If you are smart about it, you can potentially have a big impact without wearing yourself out.

3 - Learn to not take criticism of your institution personally.

Most of the feedback has been constructive or at least polite, but there have been a few that have really gotten under my skin and made me feel truly awful. The worst actually cropped up this week and was about - you guessed it - water. Not only did the student feel that we were expecting people to "drink from the sinks like dogs" but wondered how long the library was going to "ignore" the issue. This hurt, particularly because I had been trying for months to find new angles and new approaches to get management to solve this issue. Even if it was not being resolved, it was certainly not being ignored.

That was a moment when I needed to step back and put myself in the shoes of the individual who wrote the comment. From their perspective, it must have felt like we were ignoring the issue. If students see that nothing is being accomplished, they assume that means that nothing is being done. It wasn't a personal critique, just an expression of frustration at what to the students must seem like a no-brainer issue because we haven't been able to adequately justify the library's position.


I think empathy is the key to negotiating these difficult situations. If you can simultaneously empathise with both parties and truly understand their perspectives, it makes creative problem-solving and negotiation much easier. The moment you let yourself get worked up and feel personally affronted, you shut down your ability to approach the problem from different angles because now you are simply approaching it from a defensive position rather than one based on understanding.

4 - If you really believe in something, don't give up arguing for it. Keep changing your approach until you can reach a compromise.

Everyone's heard the phrase "You have to pick your battles", right? I kind of have a love/hate relationship with that phrase. It's useful in as much as it can remind you about the finite energy, finite resources or finite flexibility you have to work with, as in my previous points. But when it comes to dealing with people, this phrase makes me very frustrated. "You have to pick your battles with ________, you know?"

To my mind this just excuses the other person from having to make compromises. I'm certainly not saying I want to be adversarial over every single issue. Indeed, the most effective approaches involve seeing the issue from the other person's point of view and addressing their concerns in a creative and civil way. But I don't think that you should let "picking your battles" excuse anyone from having to engage in conversation over your difference of opinion on important issues. Nor should you let it excuse you from having a conversation that you're nervous about having. If you truly believe in something, keep trying different angles, different approaches, different solutions.

Somewhere in the depths of the UXLibs hashtag on Twitter I saw a quote from a keynote address (I'm really sorry, I don't remember which one or whose tweet it was) about approaching service design using the same rules as improvisational acting. In improv, there's the idea of "Yes, and..." This is where actors doing a scene together always build on what was said before. They never shut each other down by contradicting the inventions of their fellow actors, and they keep the scene going by expanding and elaborating on what has just been said, no matter how off-the-wall it may be. There was the idea at the UX Libs conference that this kind of environment is ideal for design because it keeps creative momentum going. Maybe some of the ideas will be impossible or undesirable in the end, but as part of the creative process it's important to run with them anyway because what sounds like a mad idea may end up being the brilliant solution you've been looking for.

If you are lucky enough to work in a "Yes, and..." environment, it's likely that you have a lot of scope to innovate. If you work in a "No, but..." environment, chances are you find it difficult to stay positive and keep coming up with new ideas. I know that I can get very emotionally exhausted by keeping my own momentum going while being told why my ideas won't work, why things won't change and so on. I'm not saying that you should ignore your own emotional well-being and plug away at things until you feel like you're going to drop, but don't let yourself get discouraged on the issues you really believe in. In that sense, you really should pick your battles - make sure you're investing emotional energy in things proportional to how important you think they are. But it's worth remembering that there are multiple approaches to every problem and it would be a shame to give up before you find one that works.

5 - Don't pay for supplies for your research out of your own pocket.

This one doesn't need much clarification. If you're doing UX research for your library, you're doing it for them. If you pay for your own supplies you'll end up feeling like an idiot later on, believe me.

UX study for a large college library

As I mentioned in a previous post, I am working on a series of UX studies over the coming months to gain a better picture of the experience of users of our library's services. I didn't want to cast the net too wide, because there is ongoing research by others on the experience of students using all of the libraries available to them, how libraries fit into their working lives etc. Also the final result of these studies will be a report that I will present to the Librarian of this college library with my recommendations for what we might do to improve user experience, so while a broader view will come into it, the focus of the study is on this library in particular.

Since there was some interest in what I've come up with, I'm posting a condensed version of the study design here and I will hopefully have the opportunity to do periodic updates on the results.

The core areas of investigation:

1)    What emotional relationships do users have to the college library?

2)    Where are we best at meeting needs/wants? Where are we failing?

3)    What insights can we gain into our role in students' academic lives?

4)    What goes on in the library?

5)    How do people relate to the spaces/services in the library?

Study 1: Feedback Wall

Based on the Speaking Wall introduced by Andy Priestner on the UKAnthroLib blog, the first phase of the study is a feedback wall in the library. This is already in place and has been ongoing since early February. I have been tracking the responses and while there have been quite a few issues that we could not resolve immediately (lack of water fountain, temperature, etc.) we have introduced quite a few new things to the library as a result, including new hand dryer for the male loos, pens on chains for the catalogue computers and we have ordered external DVD drives that will be borrowable by people whose laptops or tablets cannot play DVDs or CDs, since we have a large collection of both.

Anecdotally, I have noticed a slight increase in the number of people approaching the desk with queries since introducing the feedback wall and potentially they are approaching in a more open and friendly way (though it could be that my tyrannical campaign of saying hello to everyone who comes into the library is finally paying dividends).

Study 2: Issue Desk query tracking

On a sheet of paper, staff at the issue desk will put a tally mark under the most suitable category each time they are approached with a query. The categories are: Basic directional, Borrowing queries, Requesting supplies, Technology queries, Help finding a resource, Procedural and Other, with a blank space for noting down, if time allows, the specific query.

This study is primarily to supplement the others. I think it would be interesting to know what people are coming to a member of staff for in order to highlight what points of friction for which they are not coming to a member of staff. I wanted something really simple that would not be time-consuming for the front-line staff and the Cambridge University Library does something similar on an annual basis. I thought it would dovetail nicely with the rest of the studies.

Study 3: Student survey

Last year was the first year the library made any formal attempt to gather user feedback, which we did using a survey sent out to the entire student body. We had very good response thanks to offering an incentive for one person to be drawn at random (£25 Amazon voucher). The results were very interesting and led to a lot of small but cumulatively significant changes in the library for this academic year. This year we intend to do the same, but I am tailoring some of the questions toward specific things we want to know, like how many people know about various services we offer, how people would want to use the library if we had 24 hour opening and how they feel about non-borrowable duplicates of popular texts.

The advantage of surveys is their ability to get data from a large and broad sample of people. However, we learned last year that too many open-ended questions of that many people gives a muddy picture. In the end I spent hours and hours coding written responses in order to help them fit in with the quantitative data I was presenting. I did use specific responses to highlight the issues raised, but that data lacked depth. Therefore, I've kept open-ended questions to a minimum, though I did still want to give people the chance to leave written feedback if they did not feel comfortable using the feedback wall. The survey will be emailed to students near the end of Easter Term and once again an incentive will be offered, to be drawn at random.

Study 4: Behavioural mapping

This study comes from the lady herself, Georgina Cronin, UX Librarian at CJBS. Over the course of a couple of weeks she sat for hour long shifts in the library and observed 1) where people went, 2) what they were doing, and 3) how long they did it for. She mapped this information onto a plan of the library and created both a heat map and a coded map of different activities in the library.

This is going to be more challenging to do in my library as it is difficult to see multiple parts of the library at once. I may have to break it down into specific areas. However, I think it's worth doing because the library is not overlooked by staff so we don't really have any idea about behaviour in the library. At the very least I would like to make a heat map of where people sit most often and whether they are studying individually or working as groups, even if I'm unable to map activity and continuous flow through the library. This could give us an idea about where the most desirable seats are, how much the library is used for browsing, reading, writing and other activities respectively and whether any areas are conducive to group work.

One of the main challenges here is being unobtrusive. There is very much the feeling that the reading rooms are student spaces, such that when staff do walk through we draw a lot of attention. As a recognisable member of staff, will I be creating a distraction and/or changing the way people behave in the library? I'm still undecided about whether I want to try to do this by walk-through or by sitting in the library for long stretches as Georgina did. This might have to be a study that evolves as I learn what works in the space and what doesn't.

Study 5: Photo diary study

This is the study about which I'm most excited. It was mentioned in Georgina and Meg's class and was developed from Foster et al., though I have seen it in a few places, each time in a slightly different form. I plan to recruit about 5-8 students to photograph pre-made signs in locations they think fit the content of the signs. Several of them have blanks to fill in, so students can express frustration, boredom, annoyance, positivity, affection, and so on. The signs read:

I ____________________ THIS PLACE IN THE LIBRARY BECAUSE 

 

I _____________________ STUDYING/ WORKING HERE BECAUSE

WHEN I NEED A BREAK FROM WORK I COME HERE

WHEN I AM WORKING THIS IS PRETTY MUCH ALWAYS NEARBY

EVERY TIME I USE THIS I FEEL ________________________

THIS IS SOMETHING I WOULD SHOW A FIRST YEAR IN A TOUR OF THE LIBRARY BECAUSE

I will encourage participants to use as many or as few of them as they would like and to be as creative as they would like, taking photos within and outside the library with their own devices. As this is a more labour-intensive study for participants, each one will receive a gift card but will also be told that they have the right to leave the study at any point and dictate how their photographs are used. After the photographs are returned to me I will have a short interview with each participant to discuss what they chose and why. 

I'm hoping that because I am in contact with the Student Union president, who is very enthusiastic about working with the library to make improvements, recruitment will be a bit easier. I really look forward to seeing what people come up with on this, though.

Study 6: Storytelling/Open-ended interview

This study will be the last one I do so that I have time to build my own confidence and look at gaps in our understanding of our users. It is the most purely qualitative and the one for which I can do the least planning. Ideally, I would like the interviews to be a kind of conversation shaped by the interviewee, focused on their experience of using the library. This will hopefully generate very in depth information about how individuals relate to the library, problems they've had that we haven't found out about, how the feel about the space and the experience and so on.

Feedback spectrum: The types of feedback I expect to be generated by each study.

As you can seen from the chart above I have placed each of these studies on a spectrum based on what sort of feedback I would expect to be generated from each study. I did this to ensure that I was not looking solely at one type of data, nor at one aspect of the library experience but that I am building a relatively broad picture of library users.

The horizontal axis is adapted from the AEIOU framework discussed at the UXLibs conference and is also somewhat based on the discussion of the difference between tasks and goals in libraries. Tasks are finite and explicit, while goals are more implicit and harder to design for, but a successful library design takes into consideration users' goals as much as their tasks. Helping users achieve goals is the difference that makes some libraries great and others kind of "meh". It's much harder to put your finger on, but that's where qualitative data is especially useful.

Caveat (more for myself than anyone else):

It's important to remember to be flexible. While I have tried to design these with a spectrum of feedback in mind, I feel it's important to let the studies reveal things organically rather than forcing the results to conform to my expectations. This is my first time doing any UX/ethnographic studies so this is a learning experience for me and while I tend to like to rely on planning things to death before I do them in order to feel as prepared as possible, I need to make sure I'm open to going with the flow.

These studies are pretty much all based on the work of others so if I have failed to give proper credit I apologise.

Anyway, that's the plan as it stands at the moment! It's possibly a little ambitious for my first foray into UX studies but you don't learn anything unless you try.

UX and Ethnographic Methods for Librarians

I feel extraordinarily lucky to work where I do. I have a lot of latitude, including the opportunity to really shape my own role, I have been able to usher in pretty major changes in the culture and services at my library working from the bottom of the hierarchy and I have a pretty sweet work space from which to research exhibitions, explore avenues of outreach and publicity and keep up with what's going on in other libraries.

One of the nerve centres of the UX revolution (AKA my desk)

One of the best things about working here - and in Cambridge Libraries in general - is the focus placed on the CPD (continuing professional development) of library staff. We have great opportunities to learn and grow as a community, not least of which is the Librarians in Training courses that run throughout the year. If you're working in Cambridge libraries and haven't yet taken advantage of these, you should really think about doing so. Not only can you learn a lot from courses targeted to Cambridge librarians, but it's a fantastic opportunity to meet people from other libraries who will be your allies and resources in projects you might want to take on in future. (And they're all lovely people, it goes without saying!)

I attended a course yesterday afternoon entitled "UX and Ethnographic Methods for Librarians" taught by hippest-of-the-hip Cambridge librarians Meg Westbury (Wolfson) and Georgina Cronin (Judge Business School). Both of these women are inspirational innovators who really care about providing great services to library users and it's fantastic to get an insight into how they've done what they've done at their own institutions and for the wider Cambridge library community. The class itself was pretty much a perfect mixture of instruction and activity as far as I was concerned. I think I'm not alone in having had a really good time with it.

Introducing UX and ethnography

Meg began by defining and explaining UX and ethnography. Very briefly, UX (user experience) is a term originating with Apple, who are masters of the art. It encompasses UI (user interface) and usability (a term that itself encompasses how easy a tool or service is to navigate, learn, remember etc.), but goes beyond both to look at the qualitative dimensions of experience, emotion and cognition at the point where users interact with products or services. Essentially, it puts the focus on the individual's motivations, needs and feelings. It looks at barriers to use as well, trying to determine who is missing and why. Ethnography is writing about culture, with the worthy goal of doing so from ground level instead of top-down. As much as possible the aim is to understand the shifting points of view of the individual user without imposing assumptions or putting them in a static, statistical box. You can learn more about students from an ethnographic study with four participants, Meg said, than from surveying 100 with a standardized questionnaire.

Ethnographic UX research in libraries is growing rapidly as library professionals confront the fact that not only do we not necessarily know best about how to meet our users' needs but maybe we don't even know their needs as well as we like to think. As so-called "disruptive" technologies undermine the privileged position libraries have occupied as the sole gateways to information, how can we change our relationship to the information landscape and to our users in ways that will create meaningful, inspiring opportunities for our users? Luckily, UX research can be a lot easier to implement than more traditional research. Since you're looking for qualitative data that is not exhaustive so much as a window into a single user's experience at a single point in time, study design can be more flexible and creative. Many of the methods utilize things as basic as users' own mobile phones and post-it notes and require less planning and preparation than a questionnaire.

Methods

The majority of class time was spent looking at a few of the potential methods with Georgina, including trying them out for ourselves! First off was my favorite, cognitive mapping. The task was to draw from memory a map of your working spaces. The exercise lasted 6 minutes, and we changed pens every two minutes. This highlighted what was most significant in our minds, what was lower priority and what was most peripheral. (The order was blue, red, black, in case you want to psychoanalyze me based on the drawing below, i.e. I drew the Space Marine figure on my monitor long before drawing a representation of reshelving).

My cognitive map of the places I work. Don't judge me. I have not the skills of an artist.

Normally participants would label their maps and discuss them with the researcher, but before doing that we swapped maps with someone else and tried to interpret theirs without any labels. This clever exercise showed how easy it is to misinterpret or assume you know what something means when actually you have it all wrong. It was also interesting to experience firsthand how it is slightly nerve wracking to do the drawing but really quite fun to talk about the drawing afterward.

The other main activity was looking at observation. Georgina herself learned a lot about how users move through her library simply by sitting and watching for an hour at a time on different days. She created a heat map and a list of activities performed by library users and got a better idea of how people were actually using the space. We went out into the wilds of the University Library to try it out! It's surprisingly difficult to be discrete, especially when you're an adult in a student space. A few people were asked if they worked for the UL. Not so subtle. It is interesting how much you can learn even from five minutes. On the top floor I found that no one really stayed up there to work but rather came and got their books and left, but that there was some difficulty locating items, meaning users were backtracking a lot and looking at the signage to try to work out where their book was. There were many other interesting observations from the other participants as well. I think it's a useful method, but would work better in conjunction with something that elicited user responses as well, for instance maybe I would interview a few students about what spaces they avoided in the UL and why, or ask them to do a photo diary of places they feel confused in the library.

We received handouts at the end with a list of resources to get started (which I will copy below) and an expanded list of methods (which I won't copy, but I'm sure Meg and/or Georgina would be happy to supply if you email them). I'll be hanging on to these long-term, I think, and referring to them for inspiration.

Conclusion-y type things

After a discussion about analyzing and presenting the results, the class ended with a brief discussion of ethics, which boiled down to common sense: let participants know how you're going to use their data, and don't let incentives create an imbalance of power or a sense of obligation. When in doubt, use a consent form, and let participants know that they can leave the study at any time. Also, food-based incentives are often more than enough to make participants feel appreciated for giving you their time.

One thing I would have liked to discuss in more detail is what kinds of questions lend themselves to these sorts of studies. I came away feeling like people might just pick a study that sounded fun without first designing it to address a specific gap in knowledge. I think it would be more useful to start from a particular aspect or question you wanted to investigate, as in this passage from Nancy Fried Foster et al.:

"Our first task was to identify one trenchant research question to guide the project. The question we developed was, What do students really do when they write their research papers? Between the assignment of a research paper and the finished, submitted product was a black box that largely concealed the processes undertaken by the student. We wanted to take a peek into that box to see what we could find. We felt that this question accurately reflected our ignorance of student work habits while providing a manageable focus for our information-gathering activities." - Studying Students

I like that this passage acknowledges the ignorance of the researchers. In studying social science research methodology for my librarianship course I often felt that studies commenced with researchers already having an idea about the results in their heads. It left little room for flexibility and possibly influenced interpretation, especially when studies attempted to fit qualitative experiences into quantitative data. Statistics have their place, of course, but they don't tell a story and they certainly don't put the individual at the heart of research.

I will certainly be using some of these methods to conduct studies in my own library and have already started annotating my hand-outs with potential research topics/questions.

I'm sticking notes on so I can keep the handout for longer. Yeah.

I really enjoyed this course and got a lot out of it, most importantly the confidence that this is something very achievable and useful to do. I am excited to find out what my library users need and want, and how we can more closely align our services to those real needs. Thanks, Meg and Georgina!

Resources to get started

• Nancy Fried Foster et al., Studying Students: A Second Look. Free download at http://bit.ly/1Bj5QMp
• ERIAL Project Toolkit: http://www.erialproject.org/publications/toolkit/
• Donna Lanclos, The Anthropologist in the Stacks: http://www.donnalanclos.com/
• Cennydd Bowles and James Box, Undercover User Experience Design (2010)
• Leah Buley, The User Experience Team of One: A Research and Design Survival Guide (2013)
• Sam Ladner, Practical Ethnography: A Guide to Doing Ethnography in the Private Sector (2014)
• Martyn Denscombe, The Good Research Guide for Small-Scale Social Research Projects (2010)
• #UKAnthroLib blog: http://ukanthrolib.wordpress.com