Category Archives: mobile design

design, home school math, mobile design, modular learning, online learning, product management, user experience, UX

Hey Educational App Designers, Stop Creating Glorified Worksheets!

Leave a comment

Educational app designs need a rethink.

I am a product manager and UX designer in the software industry, and I’m also a parent of an elementary school aged child. Learning applications are a big part of our educational experience, and I find I am constantly frustrated by them. While they often have great promise and claims, and they use modern graphics and game engines, they rarely use the technology to help facilitate learning. In fact, they often put fantastic game engines around worksheets. They have spectacular characters, a wonderful environment and storyline, then for the actual math or literacy, they just display a virtual worksheet to complete. Even worse, if the user gets a question wrong, instead of showing them how to fix the problem and learn from it using the technology, they just lose points, need to find the correct answer somehow, and at worst, are unable to progress. The fun part of the app is often the parts around the actual learning, rather than making the learning part the fun part and main focus. No matter how cool and amazing the application is, if you are just wrapping that around the same old printed worksheets students have used for decades, you a really aren’t making good use of the technology.

Educational apps that don’t use game engines or a game format still tend to use game mechanics in their design to help students understand progress, facts they have mastered, concepts and activities they have tried, etc. When you know what to look for, you see the mechanics in virtually every educational app, but no matter how cool, new, flashy or exciting, they tend to devolve into learning as worksheets.

Here are three examples of math apps we have tried in the past.

We downloaded a math app, and it was a sandbox style game with customizable avatars, a rich environment to explore, and lots of clever use of music and animation. When it was time to do math work to earn points to buy things to add to your game environment, the user had to answer a series of questions, in a certain period of time. While the graphics were nice, and there was animation to help make it more engaging, it was still just look at a math equation, enter in the answer, and move on. If you didn’t reach a certain number of points in a certain amount of time, your progress was stuck. You couldn’t do anything more in the game other than wander around until you passed that level. This isn’t much fun, particularly if it is a skill you need to work on. Not only do you need more practice, but the game isn’t fun anymore.

Another math app we tried had an immersive RPG play style. You choose and customize an avatar, and your character does quests, engages with other players, and had boss battles, and other fun activities. This looks fun! However, when it was time to do math questions, you are literally taken out of your immersive environment and shown a virtual math worksheet, like the kind you print out and fill in by hand. At least with this app you aren’t punished for getting answers wrong, you just get a certain number of points to continue. However, there isn’t a lot of actual math learning going on, you have to practice those skills off screen, then come back to it. My son was so impressed with this game and loved it so much, he dedicated daily homework to develop skills to advance further. We spent three weeks of doing daily math practice and worksheets so he could master the level. While we were impressed with how motivated he was, we were baffled on why the game didn’t provide that practice. There was a little bit of support to show what the correct answers were, but beyond that, it was just doing worksheets.

The worst example of a math app with poor mechanics was one that used no graphics at all. It just had math equations, a timer, and a score. There were no visual indications of how many questions to answer, and many of the questions required off screen work, since there was nothing to do other than fill in the answer and hope for the best. To do the work to solve the problems required several minutes of whiteboard or work on paper off screen, then enter in your answer. If you had a typo, or an off by one error, you not only get a message that your answer is wrong, but your score reduces. If your score doesn’t reach a certain level, you just continue, over and over, until it reaches a certain point. You have no sense of progress, and while the app would show the correct answer, it didn’t do anything to teach the student how to do better. You get points for the correct answer, so you better come to the app with a lot of facts memorized. If you make some mistakes, not only do you not get much feedback, but you get punished. Furthermore, if you are too slow, that also affects your score, dragging the effort out even longer.

There are lots of apps that at least provide visuals and let students change their minds, but they are still mostly virtual worksheets that are trying to get students to enter the correct answer. While there is a place for that, such as dragging letters around to make words, dragging words around to figure out parts of speech, or moving objects into groups to divide or multiply, they are still not utilizing technology to help learn very much.

In math, it is so simple to design a visual calculator, and let people play around with numbers and see how that affects outcomes, and how patterns start to emerge. Once math stops being abstract, and people can play with manipulatives and see what happens, things can really click in a learner’s brain. Math manipulatives such as number blocks, Montessori boards, cuisinaire rods and more are extremely helpful learning tools, but virtualizing them, adding in animation and allowing safe exploration would be incredibly powerful. Instead of catering to learners who do well with worksheets and flash cards, learners who are struggling to understand a concept should be able to visualize the concept in various ways, play around with inputs and outputs, and see how the concept manifests itself. Not everyone can translate abstract math concepts into visualizations or numbers in their minds. Providing ways to see not just how objects and patterns interact with math, but how those concepts can be applied with virtual tools holds a lot of power. While all the technology is available to us, educational apps tend to fall back on some sort of worksheet, which only appeals to a certain kind of learner. On the other hand, virtual objects you can interact with and learn from are more engaging to every learner, and they can help people actually learn something new.

Use Technology as a Safe Place For Learning from Mistakes

What drives me up the wall with educational apps is they tend to only focus on getting correct answers. Instead, they should provide a space for experimentation. What happens if I play with addends or minuends? What happens if I multiply negative numbers together? What happens if I play with the variables and use huge and small numbers in a multiplication problem? What happens if the divisor is larger than the dividend? What if the divisor is an emoji or a letter? What does it look like if I make a word problem come alive? What happens to a graph if we loop through a huge number of values for x and y in a linear equation? What happens if I watch an animation of a huge range of possible inputs? What if the inputs are at extremes or nonsensical? Imagine how that can be quickly visualized, and different types of inputs can change the outputs, and what patterns arise from different kinds of mathematical concepts.

The beauty of virtual tools is they are SAFE places to make mistakes. You get to put in some inputs, and then watch what happens. In real life, when you make a mistake on a worksheet, you have to erase it and fix it. Virtually, there are no eraser smudges, you just change it. Furthermore, a printed worksheet can’t come alive and show you what happens when a train leaves Philadelphia at 6:00pm and another leaves New York at 7:00pm, or how many ball bearings can fit in the back of a pickup truck. Game engines with virtual tools can. Furthermore, making mistakes should be fun learning experiences, rather than being punitive. Sure, there is a point in learning where it is important to have precision and to be able to do things by hand is vital. However, playing around with technology and seeing what might happen will help students form a picture in their mind of how a concept works, not just memorizing how to get the right answer.

Actually using game engines, game design and having and understanding of different styles of game play will help people with different needs be able to learn the concepts in the app itself. Different players will have different needs, and while some people like timed tests and fact based answer seeking, others are vastly different. Andrzej Marczewski makes it easy for us to learn and incorporate Gamer User Types in our designs. For example, a socializer might want to help others learn something they struggled with, and provide a tutorial of something cool they discovered. A griefer might giggle away putting in extreme values. An explorer might try lots of different combinations of things to see how that is visualized or what virtual outcomes might be. An achiever might be motivated more by virtual rewards and determining how many and what kinds of activities to complete. There are a lot of differing user goals and scenarios, and there is a tremendous amount of knowledge and experience in the games industry we can learn from.

A number of years ago, I was asked to do a UX audit of an anatomy app. It had beautiful graphics, and a ton of fantastic information. However, it was really just a digitized version of something like Gray’s Anatomy, the famous anatomy text book. Sure, you could search, you could look at amazing graphics and click around to help you memorize, but was not using the technology to help teach. I saw two problems immediately: it was a digitized book or worksheet, and it was static. Anatomy in living organisms is not static. A living organism has different states in their body at all times. For example, there was no use of technology to show oxygenated vs deoxygenated blood in a circulatory system, or to simulate illness, pathologies, or other things a med student needs to do to apply their anatomy knowledge. Furthermore, testing was fact based. You needed to memorize facts using the app, and then state those facts in an exam. The learning was about reading and looking and memorizing, not experiencing. When you are a medical professional, one of the most important sources of learning is from mistakes, or from failures. A patient doesn’t respond well either due to a lack of knowledge or the wrong treatment, and you learn what not to do.

My design concept was to digitize that virtual patient experience, as sort of a medical study Tamagotchi. Instead of memorizing a virtual anatomy text book, why not have a virtual patient to keep alive for a semester? Sure, you have the required anatomy to understand and commit to memory, but you have a simulated patient who can have certain illnesses, pathologies or states to manage. It sounds crass, but if your virtual patient dies a lot of times, you are going to learn a tremendous amount from that experience that you can use in the real world. It is much safer to learn and fail and see what happens to your virtual patient, rather than memorize and get a poor score if you get things wrong. If you can fail virtually and learn from it, that has a lot of value. I would prefer to play and experiment and be rewarded for learning from mistakes, rather than memorizing text books facts and being afraid to fail an exam. Exam scores have real world consequences, but playing around in an app and having fun, piquing curiosity to explore “what if” scenarios, or having instructors throw you challenges to keep your virtual patient alive is something we can absolutely do with computers that we just can’t do with dead tree text books.

Another area to learn from video games is how they treat failure. To make a game engaging, failure is part of the game, not a punishment. In popular games, their designs never make you feel lost or dumb. You feel like a super hero, and when things go wrong, you can recover and try again. In fact, many games make the failure part incredibly fun and rewarding. Who doesn’t want their avatar to scream in a ridiculous way and burst into flames if they fall off a balance rope in an obstacle course? Some failure modes are so fun and hilarious that people spend more time crashing their characters than completing tasks. Even games that are extremely challenging and are designed to be frustrating are engaging and use that frustration and failure to encourage people to try again. You aren’t left feeling stuck and dumb, you feel like you need to try again, just one more time. Furthermore, if your character crashes and burns, you just respawn and try again. You aren’t stuck unable to play without doing a lot of work outside of the game to continue. The game helps you succeed, and if you are really stuck, game communities are fabulous places of sharing knowledge and helping each other.

The more I experience educational apps with my kid, the more I see that educational app designers completely miss the power of virtual technology and learning. They should design the apps around experimentation and reward failure as a part of learning, but they end up digitizing worksheets. They expect people to know facts, they don’t help people pique their curiosity in a safe way. They have an extremely narrow view of learning and teaching. Why don’t they support inquiry and experience? Why do they just duplicate books and worksheets, even when they have a fancy MMO or RPG engine around the learning? Virtual learning environments themselves are fantastic places to do whatever you want to learn. Where else can you safely find out what happens if you feed something poison, or if you fly the rocket into the ground, or you play around with your math question variables, or if you rearrange your words in a nonsensical way. These are pretty bad ideas in the real world, but great learning experiences in a virtual one. Plus, mistakes can help you learn, and they can be fun and silly. Laughing at a ridiculous mistake on a math concept and visualizing the carnage is a much more effective learning technique than getting a long division problem wrong, after you spent ten minutes solving it off screen.

Using the technology to just digitize the printed worksheets completely misses out on this important approach to learning. Sure, at the end of the experimentation, you want the student to have knowledge and skills and to have learned, but we have game engines, and graphics and powerful machines that can be used to learn what we need them to learn, and we instead just give them worksheets. And in many cases, the worksheets are even worse than a printed one.

Bottom Line: Let students play with the concepts you are trying to teach and let them succeed and fail in a safe way, using everything technology affords us. Stop punishing learners for making mistakes, let them make mistakes and explore the outcomes virtually. Stop taking dead tree technology, digitizing it, and rewarding people for getting the correct answers and calling that an educational experience. Use the technology to show, tell, demonstrate, play with and really get a solid grounding in the concepts without real world consequences. That is the differentiator with learning with technology: you have limitless access to information, and tons of rich tools to virtualize problem solving and learning in stunning ways. Provide structure and opportunities to learn, don’t just expect people to write an answer on a worksheet. Give them more.

UPDATE:

April 24, 2024

I was reading this article about educational apps: The 5 Percent Problem: Online mathematics programs may benefit most the kids who need it least, and there are some thought provoking points. This quote in particular stood out: “…the programs may have been unintentionally designed to fit high achievers better, says Stacy Marple, a researcher at WestEd who has studied several online programs.”

Put another way, if you design apps that expect learners to already have mastery, they will tolerate your virtual worksheets because they can easily enter in the answers. They have the knowledge, skills, and confidence to grind away to get back to the fun part after the get the math or literacy “worksheet” completed. For learners who don’t already have mastery, they will be frustrated and stuck, because there aren’t mechanisms in place to help them safely learn, to build their understanding and confidence, and to actually help.

books, mobile, mobile design, mobile testing, product management, Tap Into Mobile Application Design, Tap Into Mobile Application Testing, usability, UX

New Book Published: Tap Into Mobile Application Design

2 Comments

After a long delay due to some health issues, I have finally finished the first version of Tap Into Mobile Application Design. This book is only available in PDF and epub electronic formats which should work on most e-reader apps on most mobile devices. I used Leanpub as the publishing platform, as with Tap Into Mobile Application Testing. I like this platform because it’s faster for me to get content out there, changes and updates are instant, and customers can download updated copies for free.

Tap Into Mobile Application Design book cover

This book is long and detailed, a result of me trying to capture most of my thoughts on designing software for mobile apps. When I started the book, I had planned for something much shorter, but as I worked through the content, it felt abbreviated and overly simplified. In Tap Into Mobile Application Testing, I made a couple of mistakes trying to overly simplify complex technical issues with regards to wireless technology. I updated some of the content to be more accurate, but I didn’t want to repeat that problem with this book, so I went deeper in some technical areas.

To help illustrate the challenges we have on these projects, I decided to use an example app project in the book. This helps to ground the content, moving from initial idea, to a full user experience design process and ending with user testing. The example project helps illustrate what a real world project can look like, but with the benefit of time I was able to capture many design project issues, rather than the few you encounter on a rapidly developed app. You get it all, including the positives and the negatives of the example app.

Furthermore, the context of app design changed as I was writing, and I felt I should capture some of those changes in the book as well. The legal landscape has changed, and there is a much better awareness of ethics and the long term effect of our designs on people. With the benefit of a side project to use as the example in the book, I was able to capture these issues as they happened on that project. The project had to adjust, and that is reflected in the book. Unexpected issues are common on mobile projects, and the example app shows how we adjusted. Initial attempts often fail due to oversight, legal rulings have an impact, and the timing of what you do on a product is crucial.

The book is also longer because it doesn’t just follow a happy path. There are lots of great books out there that fit that model. Instead, this book covers false starts, changes in direction and a completely reworked interaction design. That’s right, I cover how we almost went to market with one design, hit a snag, and completely redesigned the example app from the ground up. It’s difficult to capture the non-linearity of design in a book, and that results in some awkward flow and a couple of extra long chapters. I apologize for that, but I had hoped this would be an honest and detailed account of what can happen when you are creating an app.

I have also created a book bundle, combining both my “tap into” books called Tap Into Mobile Apps, where you can buy both the books for about the same cost as the full price of either book. The books are very different, but are complementary. In Tap Into Mobile Application Testing, the reader follows Tracy, a tester who is learning about mobile testing approaches. In Tap Into Mobile Application Design, the reader follows an example mobile app project called “Reporter” throughout the book. The design book is more heavy and dense content-wise, and that is reflected in the tone. The testing book is lighter in content and tone and an easier read. Both books cover technical issues to help inform your work. The combination of designing for or testing for people in social contexts, with a deep understanding of the technical underpinnings of the technology, within real world environments is my core differentiator. When I help teams develop that three pronged approach themselves, they build better software and have happier customers.

Both of these books represent my approach to working on mobile apps, which people can utilize as they see fit. These books aren’t for everyone. They are long and detailed and don’t provide easy answers. What they do provide is context and details that are important to understand on mobile projects, especially when you are having trouble. In spite of it representing a more difficult approach to your work, Tap Into Mobile Application Testing has been used by people all over the world, and influenced many mobile projects. It was highly praised when released and even now people contact me to tell me about how much it helped them. People still use it, they still talk about it at conferences and on projects, and several years on, find it relevant and helpful. I hope as many people find the design book to be as useful as they found the testing book.

business, customer service, design, mobile, mobile design

A Bad Mobile Experience is Bad Customer Service

3 Comments

As a travelling consultant, I get insight into a lot of organizations. One theme that I see over and over is a lack of understanding about mobile experiences among decision makers and technical teams. When it comes to interpersonal or written communication, companies have sensible rules and practices about making that exchange great. You’ll hear or read slogans about treating the customer right, or to go beyond and satisfy and impress them. Most organizations have great alignment on that message, and when I walk into an office, I am greeted by friendly security and front office staff who smile, are pleasant and help me get to where I need to go. They want all of us to have a great first impression, so they invest in human-friendly lobbies with art, comfortable furniture, and great lighting. This extends to email and voice communications as well. People try to be professional, pleasant, and make me feel welcome as a customer or consultant. However, that is rarely my first impression of that organization. For me, and millions like me, I get my first impression of that organization through their mobile experience.

In many cases, that mobile experience is the opposite of their interpersonal customer service. They fail to realize that a bad digital experience is just plain old bad customer service. It is the virtual equivalent of a rude call center employee, or an office administrator who tells me to go f— myself when I walk through the door. Instead of feeling valued by the organization, my mobile experience feels like they don’t care. They make me feel confused (what does this company do?), they make me feel frustrated (I can’t solve the problem I need to sort out!) and that they don’t value my time (why do I have to gesture so many times and spend so much time to do a simple task?)

These organizations would be horrified if I was flipped off when I walked in the door, but they have no problem making me feel just like that when I try to interact with them virtually on my mobile device. And they don’t seem to have a problem with it. On the other hand, organizations that realize great customer service extends to digital experiences have alignment on their customer-first values. Instead of worrying about mobile frameworks and handsets and technology development, they seek the solutions that provide a great customer service on virtual mediums. Thankfully, that means we have a lot of examples on how to do it well.

Here is an example of contrasts: airline mobile experiences. I won’t name names, but I will describe 3 different mobile experiences and the technology they are using to either create great customer service, or do the digital equivalent of flipping us off.

I travel a lot, and when I am travelling, I need the following at a minimum:

  1. Ease of use (so I can get my information quickly when under pressure on a trip)
  2. My booking details stored: confirmation number, flight numbers, seating details (this eliminates printing out paper that often gets out of date)
  3. Access to my booking details when I don’t have network connectivity (when you travel, you are frequently without wifi or cellular data, particularly if you don’t want to purchase an expensive data plan during a short trip)
  4. Up to date flight information when I am connected to a network (so I can get out of jams when there are delays or cancellations)
  5. Ability to book or modify or cancel flights
  6. Simple contact to customer service if all else fails on my own

Let’s look at three different mobile experiences from three different airlines:

  • App 1: a native mobile application
  • App 2: a hybrid mobile application
  • Web 1: mobile optimized web-only access

How do they stack up?

App 1 (native):

  1. Ease of use: incredibly easy to use, and it takes 2 gestures to get my current flight information. It takes about 15 seconds to tap the app and get all the info I need to fly
  2. Flight details stored: all my flight information is stored so I don’t need to search elsewhere
  3. Offline access: this app not only displays all my information when I don’t have networking or wireless access, but if I am not connected, it provides a warning message and tells me the data may be out of date. This is useful in cabs, in areas of an airport without connectivity (such as in security) or in an aircraft while in airplane mode so I can check my next flight if I have a connection
  4. Information updates: this app lets me know of gate changes, delays, and even lets me check on in-bound aircraft status and other dependencies. When there are problems, I and others with the app know before official announcements are made, which gives us a jump on the competition for rebooking or making alternative arrangements. This app has helped me get to a client site or get home on time on a number of occasions.
  5. Flight bookings: this isn’t easy to do on mobile devices, but with a few taps and gestures, you can get the job done without feeling frustrated
  6. Easy contact to a real person for help: one or two taps and I am speaking to a customer service rep who can help with a problem I can’t sort out with the app

App 1 overall customer experience: great!

This app has really improved my flying experience and I am growing to like this airline more over time.

App 2 (hybrid – combo of web and native technologies):

  1. Ease of use: not bad, but it is a bit clunky and doesn’t respond as smoothly to input gestures as the native app. I can’t zoom in on some screens, and on others, it takes me out of the app to the website, which is irritating. It feels quite slow, but I don’t have to do a lot of inputs, I just feel like I am waiting
  2. Flight details stored: This is a thin-client app (not taking advantage of local storage that hybrid apps provide), so I have to store my booking reference in a note, leave the app, copy it from a note, paste it in, and then wait for it to load my details. This is frustrating and takes a lot of time. There is no excuse for overlooking this with a hybrid app.
  3. Offline access: The app will not function without a good wireless connection. That means it is useless in many areas of an airport. If I want to check my flight number for a connection while the app is in airplane mode, it crashes. I have to take screenshots of details when they are available as a workaround.
  4. Information updates: The app does not provide updates. I have to navigate to the airline website or airport website and search for details on the flight. This is too time consuming and is frustrating. I get no information advantages and have to get in line with everyone else if there is a delay or cancellation
  5. Flight bookings: Not bad. Nice interface, and few gestures to get the job done. It just feels slow compared to a native app.
  6. Easy contact to a real person for help: I have to do a lot of navigating, and then I get redirected to their corp web site which is slow, cumbersome and frustrating

App 2 overall customer experience: poor.

Every time I use the app I get frustrated and I project that frustration on the airline. In one case, I was having trouble checking in at a kiosk, so I opened up the app. It gave me a cryptic error message that made me feel like my flight had been cancelled. I had to stand in a line up for a half hour to find out from a human to look into the problem and sort things out. This app requires workarounds (save my booking reference in a note or search for an email confirmation and screen shots of up to date data) to be even useful when I fly. They chose hybrid to get cross platform support, yet they don’t even take advantage of the affordances of a hybrid app.

Web 1 (mobile web site):

  1. Ease of use: horrible. I have to open a web browser, and to just enter in their URL is over 20 taps! I probably don’t have 20 taps in a month of using the other apps in total. Once I enter in their URL in my device and fix typos, 30 seconds has passed. I am then redirected to a mobile optimized web site. I can’t get both my flight status and my booking information in the same place, so to get the same information I get with 2 taps from a native app takes me over 50 taps and gestures and can take 2-3 minutes, under ideal conditions. Their image carousel on each page takes forever to load, and gets in the way of my interaction.
  2. Flight details stored: As a web app, there is no local storage, so again I have to store my booking reference in a note, leave the web browser, copy it from a note, paste it in, and then wait for it to load my details. This is frustrating and takes a lot of time.
  3. Offline access: None. I can’t do anything without a good wireless connection.
  4. Information updates: No updates. I have to go to a different website for that, and tap away on the URL, and experience the same pain.
  5. Flight bookings: Not bad. Nice interface, and few gestures to get the job done. Again, it feels slow compared to a native app and there are some inconsistencies from screen to screen.
  6. Easy contact to a real person for help: I have to go to their corp web site, which means I usually have to find the option to go to their full web site (a tiny URL at the bottom of some of the mobile screens.

App 2 overall customer experience: Rage inducing.

When this app lets me down in an airport because it takes too long to get anything done, I look longingly at the competition counters across the hall. This experience is beyond useless and it causes me to lose faith in the organization as a whole. If this is so painful, how is the flight? Also, I found that staff members have very little faith in the mobile experience. Even though I had a PassBook ticket that was valid, they reprint a hard copy when I try to board the aircraft because they “don’t trust the mobile app”. PassBook integration is one thing they do well, but people get so frustrated with an experience that doesn’t fit the needs of customers and staff on the front lines hear about it and try to adjust.

So whose app do I use the most? App 1 of course, and I also fly with them the most. I get a great customer experience with their mobile app, and they also do a decent to please us in their interpersonal interactions. The airlines that supply App 2 and Web 1 actually have better interpersonal interactions – they are friendlier and more helpful, but the mobile digital experiences are not convenient, so I don’t fly with them as often. This has less to do with the technology they have each chosen, but in how they have chosen to implement it and whether they are making technical decisions or customer-serving decisions in how they create mobile experiences.

A great customer experience that enhances convenience goes a long way. In fact, I will put up with less than ideal interpersonal interactions with a company if their mobile experience is convenient and makes my life easier. Next time you are looking at your mobile experience, no matter what the technology, ask the question: are we creating a great customer experience with our technology as well as with our people? The extent to which you use mobile technology to address that means much more to your customers than just “going mobile” and putting something out there. In some cases, no mobile access is better than one that makes people hate your company.

In an upcoming post, I’ll talk about ways our hybrid and web-only airline friends can improve their mobile customer service.