Flow Model

This is the current work practice flow model of our user roles within the CRC:

Work Practice Flow Model

This is our initial work practice flow model. It encompasses the broader view of the CRC information system.

Our proposed system doesn’t alter or replace any of the current work practice flow, but rather provides roles with more avenues for communication. Since we do not alter the communication with the outside world, we decided to focus on the individual roles.  Here is a zoomed in view of the center portion of our “before” flow model.

Work Practice Flow Model Zoomed In

This the portion of the work practice flow model that would be altered by the community jukebox.

We decided to split up the information into three different categories–private information, public information, and non-professional information.  From our interviews and observations, we noticed that some information avenues are completely absent. For example, the office never shares non-professional information to the students or faculty. This means that the funny pictures or interesting articles that could be relevant to the CHCI community stops at their office. Here is the flow model after introducing Community Jukebox (CJ–labeled as SJ, which was a previous name.)

Work Practice with CJ

This flow model shows how the community jukebox would alter the flow of information and change work practice.

With CJ, public information and non-professional information can be shared equally between all user roles. Hopefully, this would raise not only awareness of what others are doing but also gives an avenue for building personal connections and overall community.

Hierarchical Task Inventory

3DUI Research Website

For this assignment I investigated the usage of my research website, and modeled the task structure below. I have experience using this system, because I was the person who designed the system. There are two roles, the students and the admin / faculty. The students are technically a subset of the admin / faculty but I modeled them as a separate branch.

Hierarchical Task Inventory

Hierarchical Task Inventory for 3DUI Research Website

Hierarchical Task Inventory for 3DUI Research Website

CS5714 Usability Engineering – Table of Contents

Project Posts

Project 1 – Client and topic
Project 2 – Contextual Inquiry and Contextual Analysis
Project 3 – Requirements and modeling
Project 4 – Design
Project 5 – Prototype
Project 6 – Evaluation and reporting
Project 7 – Final presentation

HW Posts

Homework 1 – Design Analysis
Homework 2 – Modeling, HT Analysis
Homework 3 – Persona
Homework 4 – Sketch of phone usage
Homework 5 – Ideation and sketching
Homework 6 – Affordances

Why I love the new SC2 Inteface

Overview and UI Goals

Starcraft 2 (SC2)  is a real-time strategy game, set in outer-space, where players control one of three races in battle. Players have the option of playing against both the computer and other human players in many different ways. The main goal of the main UI is to allow people to quickly and correctly select the play style that they are interested in. Here is a list of the play options currently available and whether they count towards a player’s public rank:

  • Campaign single player – unranked
  • Multiplayer with computer and/or human allies vs computer and/or human opponents – unranked
  • Mulitplayer with two, three, or four human allies vs the same number of human opponents (1v1, 2v2, 3v3, 4v4)- ranked or unranked
  • Multiplayer with multiple human opponents every person for themselves (FFA) – ranked
  • Arcade multiplayer with multiple human opponents in a campaign style of play – unranked

Heart of the Swarm (HoTS) Beta Interface


Screenshot2013-02-03 15_46_47

Heart of the Swarm landing page

As you can see from the list above, the the player is basically split between single, multiplayer, and arcade play options. Therefore, it is a reasonable approach to make the initial action of the player to select what type of play option they want so they know all future decisions will be in that context. In the beta, these choices are presented immediately to the player on the landing page along with important news, and chat rooms (which helps facilitate group play and community building).

Screenshot2013-02-04 16_04_54

Wings of Liberty landing page

Although this may seem intuitive that is not the case with the current incarnation of the UI. In Wings of Liberty (WoL, the current expansion) the player has only quick access to “Starcraft” (a terribly in-descriptive button) or “Arcade” from every page.  The majority of the landing page is taking up by the player’s current campaign mission and a little bit by news. In order to get to the other play styles, players go to the “Starcraft” screen, which lists all the single player and multiplayer play options. There is also a quick menu, which has a list of play styles and their hotkeys listed. I personally don’t use this menu as often as I should, and find myself going through three pages to get to the menu I want rather than two as I do in HoTS.

Screenshot2013-02-04 16_05_01

Wings of Liberty Quick Menu

Although HoTS doesn’t have single player mode available, since it is still in beta, I hope they don’t automatically place campaign information on the landing page. There are lots of players who don’t even play the campaign and simply play online. The multiplayer community is strong and competitive enough, that I would hope that the landing page would serve as quick launch for a player’s most recently played mode. For example, if the last play option I played was 2v2 multiplayer, I would want to see that on my landing page with my default settings (race, map preferences, etc). That way, the game would feel more tailored to my play style and would require less clicking around to get to the features I am interested in.

The revamped multiplayer menu system

Since HoTS doesn’t have single player, I am going to look at the multiplayer menu system only. The first important thing to note is that HoTS makes a big terminology differentiation in terms of multiplayer that was once obfuscated from the player.  They have divided the multiplayer option into “Matchmaking” and “Custom Game”. This simple terminology change is brilliant, and increases the usability for new players. First, I will discuss the “Matchmaking” menu then go into why this change is helpful.

Screenshot2013-02-04 17_18_29

Heart of the Swarm “Matchmaking” menu

The “Matchmaking” menu brings some much needed changes to SC2. In particular, it lists all play styles in a way that doesn’t require players to changed pages, much like a Javascipt navigation menu in modern web design.  With the new page layout, players can see all play option at the top and see the details of a selected option at the bottom.

Screenshot2013-02-04 16_05_06

Wings of Liberty “Starcraft” menu

In WoL players first have to go the “Starcraft” page from which they can choose which play option they would like, which loads on a new page.  This means that if players wanted to play quick match, yet they accidentally clicked cooperative game, they would have to select “Starcraft”, wait for the page to load, and then choose the right option. This new way has three great advantages. The first is for novices who may not know what all the play options mean. They can quickly check out each option without have to go back to the entire listing. The second advantage is that current players won’t have their workflow completely changed – they select multiplayer then they choose their play option. (The menu time to get into the game hasn’t changed for established players.) The last advantage, is that players have all play options constantly visible to them, which may encourage players to try different play modes. In WoL, if a players breezes through the “Starcraft” screen, they may never really take the time to check out the other play options. Where as in HoTS they are hanging out up top saying “try me”. One thing I do like about the quick match screen in WoL is the very graphic representation of all the races. It seems a bit more polished than the HoTS button interface (I assume HoTS will have something similar once the beta phase if finished).

Another great change to the play option listing, as you may have noticed, is the names. Finally, we have more descriptive names which makes it easier for novices to know what exactly they are signing up for. In WoL multiplayer, there is cooperative, quick match, and custom game play options. Cooperative, has now been changed to versus A.I., which is more descriptive in many ways.  For one, quick match can be cooperative, so the name cooperative is misleading. The main difference in cooperative mode is that players are playing against the A.I., so why not call it that. Quick match, which is a ranked style of play, has now become ranked and unranked. Now you may think, why has it been changed to both, wasn’t it only ranked in WoL?  The explanation for this goes back to the “Matchmaking” vs “Custom game” terminology.  Quick match got its name from the fact that a player doesn’t have to choose a map, game setting, or find players manually to play. In WoL this ment ranked ladder play, where players are placed into leagues and compete against similarly skilled players. In HoTS, since the initial choice of “Matchmaking” has made it clear, a more descriptive name of ranked or unranked can be used. This simple change in the parent level has made for a new feature (unranked quick matches) as well as cleared up a very vague terminology from WoL. This leaves custom game, which has been kept the same as before but one level up, which I think works better than having it with the rest of the multiplayer games. The reason being that the majority of players probably play games where SC2 matches the opponents automatically. Custom games are usually used by professional gamers in tournaments or in practice, since it is the only way to ensure their opponent.

Next Match PLease!

Screenshot2013-02-03 16_04_58

Heart of the Swarm post game review interface

Another great feature of the HoTS multiplayer UI is the post game screen. In SC2 players have to master 4 areas to be successful: economy, production, army control, and overall strategy. Therefore, for players to get better there has to be some type of post game analysis that helps them identify their weak points. Economy and production post analysis reviews are usually displayed in graphs or stats about how much money was in the bank, how much was collected, and how many units were produced. This is pretty straight forward in terms of seeing where a player was lacking. Army control and overall strategy however are much more difficult.  The only way to do this is through watching replays of previously played matches and determining the point of failure.

Screenshot2013-02-04 16_37_39

Wings of Liberty post game review interface

HoTS post game review provides players with not only the graphs and stats from WoL, it also provides a direct access to the replay – a much needed feature for almost all players. Even more excitingly is that the “watch replay solo” buttons turns into a “watch replay as a team” button if it was a cooperative game.  That way players can control the same instance of the replay so they can better discuss their results. This team replay feature doesn’t exist in WoL so it is unfair to compare the two in terms of functionality, however, providing this common ground among teammates is brilliant. Another great feature that was added in HoTS was a subtle way of alerting players when they had beaten their previous average.  This is a great way of showing players when they are progressing, so they can adapt strategies for future success. Another tool to assist in understanding strategy is the “map info” button, which provides the player with details about the map the match was played on. There is community feedback, and chat rooms associated with each entry to discuss strategy and exploitable aspects of the map. Last but not least, I love the “Play again” button. As many avid SC2 players will attest, it is very difficult to only play one game (especially if you lose). The “Play again” feature assumes players are playing the same type of game and simply re-enters them into the matchmaking queue immediately.

In WoL, the post game review UI is very minimal. There is all the stats information, minus the better than average alert, but the only way to get to the reply is to go to the replay screen (the camera button under “Arcade”) which list all the replays, then select the last game. This is a very clunky interaction that hinders people from looking at their replays, rather than encourage them. The other annoying thing is when players want to play again, they have to go back to the quick match screen, and select find match again, even if they want to play with the same options as before.

Bringing people together

As I mentioned earlier the main HoTS landing page has a section dedicated to group chats. Although the chat system is similar to other main stream chat clients there are two ways people can find one another. The first is by knowing their username, whether through asking another player in real life of having played with them in game. The second is through group chats, which are basically chat rooms with a common interest. Since directly connecting with real life players, or previously played teammates is much easier, I believe the HoTS interface is looking to attract more people to the group chat functionality. The SC2 community is quite vibrant, so by adding emphasis on group chat through the landing page, I believe more people might participate in deeper conversations in game rather than through message boards alone (this is also supported though new features where groups can post news and show rosters of all members).

WoL obfuscates group chat by not only having it is a separate button next to the buddy list, it also requires players to know the the group name to join. By having highlighted groups and richer group pages, I hope HoTS will begin to foster the community in-game as much as it has out-of-game.

 Wrap Up

The HoTS interface brings some great and needed changes to SC2. I am glad to see Blizzard (makers of the SC2), working as hard on the interface as they have on all the new units and levels for this expansions. I hope to see the interface become more polished with added bells and whistles once the game is officially released and I look forward to the third expansion in the coming years!

Project 1: Client and Topic – Art Jukebox


Virginia Tech Facilities Services aims, “to provide state-of-the-art, design, construction, operations, and maintenance of university facilities, grounds, infrastructure, utilities, and transportation in a customer-focused, cost effective and sustainable manner.”


To support VT Facilities Services in this aim, to help build community around Virginia Tech, and to engage community members in casual interaction to facilitate a feeling of ownership of and belonging to the university, we will design (the interaction for) an Art Jukebox. The primary users will be diners in Virginia Tech dining halls, patrons of Virginia Tech recreation centers, and students in their respective Department’s public spaces.

Concept Statement

Art Jukebox (AJ) is a distributed presentation system, that will be installed in dining areas, recreation centers, and departmental public spaces at VT. Patrons will engage and interact with each other by contributing different media, from the web or their own collections,which AJ will display. Each area (particular recreation centers, dining halls, etc.) will have a “network” of display thats will enable multiple individuals to enjoy the same content. Much like a jukebox, users will make their request for content and the system will schedule the media that it will display. However, for AJ, the requests are made via web-enabled devices. Audience members can vote / indicate they like certain content which will have affect on current and future scheduling. Art jukebox builds community while providing patrons an opportunity to entertain themselves and each other.

Technical Summary


There will be multiple instances of Art Jukebox (AJ) around Virginia Tech to serve multiple micro-communities; however, for clarity we will describe only one, to be located in D2 Dining Hall, in this summary. In D2, AJ will be composed of two large, projected displays, situated on either side of the the middle division.There will also be one, mounted, tablet device at each table, and additionally one mounted beside each large display for submitting content. The large displays will only support playback. The tablets will support submission of content as well as display and voting.


Content Submission and Display Request

Users will use the AJ app on one of the provided tablets or their own device to submit content (via upload or URL). The system will store all content submissions so that future requests might re-submit previously enjoyed content.

Display Options

Each display will have a progress meter for the current content. Images will be displayed for a set amount of time, media that has inherent length will be displayed for its duration. A hide-able indicator will show the number of votes for or against the current content and there will be controls to mute and to vote.

Content Popularity

Since content uploaded will be viewed by everyone, content popularity will be decided based on the number for votes received by the audience. Audience members can vote (like or dislike) while the current content is playing from a provided tablet or their own device. Content that receives many number of dislikes will stop playing and the next in the queue will get its turn. If there are no content requests in the queue currently, the most popular content (most likes) will be played by AJ until a new request arrives.

The future of 3DUIs

Reflecting on our class discussions and projects here are a few things that came to mind about the future of 3DUIs.

Are 3DUIs easier to create for entertainment ?

Most of our project (which we think are cool) are related to fun.  The application specific designs we thought of in the beginning either didn’t have the “wow” factor or seemed to be rehashing of previous techniques.  Does this mean that it is easy to design cool interfaces for entertainment?Entertainment is also the most prominent place for 3DUIs currently (gaming interfaces in particular).  Does this color our perception of what we think the mass will think of as cool?

How to design cool 3DUIs?

One way of doing this by creating a new piece of hardware (Multitouch film). Another way is to take something existing and pushing it further (kinect fusion). The last way is 3DUI-fication of a current 2D interaction/interface.Our class focused more on the last way. Why? Is it because of the time constraints of the class or is it because we want more engaging interactions in our current interfaces?

Is it still possible to innovate in 3D UIs?

Through our class experiences and ideas I think there are many areas that can use 3DUIs.  In particular there were many things that we couldn’t accomplish yet exactly because the technology wasn’t there.  This is a great sign because it means that moving forward we will have new things to try and implement. One important theme we take advantage of in our project is that although not all the process are automated the user gets the effect of having a world they can interact with.  It is important to realize that prototyping and working with ambiguity in terms of sensor capabilities is important to push the field further.

Are the innovations in 3D UIs today different than they were in the past? If so, why?

I think that 3DUIs of the past and today are different because we now have more physical tools to assist us.  Before, there were no HMDs, CAVEs, Kinects, and Moves.  Graphics programming was done from scratch using OpenGL or vector graphics.  Therefore the focus seemed more about how to capture 3D inputs.  Now 3DUIs are focused more on the interactions.  In particular, we see projects that look at the interplay between virtual and real objects, physical expression, and fun.

How do new technologies spur innovation in 3D UIs?

New technologies played a big role in our projects.  Most of the projects in our class used the kinect not because there weren’t other ways to accomplish the same motion capture but because it is new.  By using the kinect, it not only adds to the novelty of the interface but its form factor allows others to experience our creation.  Therefore, we can say that new technologies have the capability of creating new techniques (modeling with the kinect, multi-touch with capacitive films, etc.) or bringing techniques to the masses (kinect, move, retinal displays for AR).

What design philosophies, inspirations, or methodologies can lead to innovation in 3D UIs?

The most important thing is baby steps.  You can go the route of creating a new hardware for an interesting technique but there is so much still left to explore with what has already been created.  It is important to understand that technology shouldn’t be the limiting factor when creating a new 3DUI.  The real limit is the designer’s imagination.  With this in mind inspiration can come from anywhere: Current games, daily tasks, things that you imagined doing, or new ways to approach a problem.

When we use new technologies or philosophies, are we ignoring the design principles and techniques developed in traditional 3D UIs?

We can’t help but be influenced by what we have been previously exposed to.  Not to mention we still use traditional 3DUIs as a measure for comparing our techniques.  That being said we need to look outside traditional design principles in order to move forward.  We currently don’t have a perfect understanding of 3DUIs, therefore we shouldn’t have design principles that are set in stone.  Eventually there may be a standard (like the WIMP interface for desktop interaction) in which we can have these types of concrete do and don’ts.  Until then, we need to keep breaking the rules :D

Captains Log 22.03.2012

The Game

Unity has a great offer allowing free iOS developer license available (more details here).  So I have had the pleasure of developing our game Insane Llamas using there software.  It has some great features that allow for easy scripting, editing and debugging.  This is a much welcomed changed from some open source projects that we were using because Unity has a large community base and good documentation.

So far, we have functionality built in for launching Kiwis based on camera position, destroying llamas,  and rigid body physics.  The next steps are to integrate PTAM to the camera, work on the models, and setting up the real world environment.

The Environment

We have also found a room that we can use for our final environment.  We can add as much real world clutter to give a fun and challenging setting for our great llama battle. Now the question becomes how big of a volume can we save with KinFu.  This will be the determining factor of how big the virtual objects will be and how intricate the level can be.

The Prop

We have also finished our slingshot prop!  After much work at Dr Quek’s workshop we have a light weight and protective design for the iPad.  The next step will be painting some cool funky colors.

iPad troubles!

So it seems that the iPad doesn’t suport having both the front and back camera capturing data at the same time.  This really puts a halt to our proposed 3D interaction.  Fortunatly, there are some cool people doing work in our same area and have posted a neat solution that we can take advantage of.

Now instead of using vision to determine where the slingshot will be launched we will use these stretch sensors instead.

These awesome sensors are made from conductive rubber and change resistance as it stretches!  This will allow us to have position determined through mechanical means instead of using the front camera.  Now the only thing that we need is a way to transmit the resistance measurements to the iPad.  Enter Redpark, a development company which has a cable that connects an arduino to the ipad.

Now our launching setup looks like this:  

We will use a lillypad arduino to perform all the analogue input reading.  We chose this sensor because it is very thin and will be less encumbering to the user / case space.  This will connect and be powered by the iPad via the Redpark cable.

The stretch sensor attaches to the case so that it can detect the release from any quadrant of the ipad face.  We think the best way to do this is by having 4 pieces of the rubber sensor.  By knowing the lengths of each leg we can determine the relative position of the user’s pinch and release.


Launching Kiwis aka Fun with OpenCV

To create our cool 3D launching interaction we are using computer vision to capture finger positions. The easiest way to do this is to track the fingertips using colored markers.

The only downside to this method is now the user will need to put on some sort of colored marker in order to interact with our system (or paint their nails / fingers).  For our application we will probably use the ends of colored latex gloves to provide the tracked color without the mess.

The pros to this approach is more robust tracking of the fingers which seems to be pertinent for this high stakes game situation (we don’t want the Insane Llamas to win do we?).  Although there are compromises that can be made (open hand and closed hand gesture recognition would be easier than marker-less finger tracking) we think that this approach will provide the best game experience.

To track the fingers we will be using OpenCV which has a port for iOS devices.  With some sample code from AI Shack I was able to get a prototype working for tracking two colors; in this case red and yellow.  In the image below you can see the paths of the objects’ center of mass and the threshold used for determining the objects.

The next step is to determine pinch and release events for the two colors. Once that is complete we can determine trajectory and force for our Kiwis!

Project Insane Llamas

Before the Wii, games were set in 3D environments with limited 2D controls. Now we see more full body gaming and natural motions with the Playstation Move and Microsoft Kinect. Mobile gaming is heading in this direction, but the majority of mobile games still follow traditional game design. We look to give the player a more interactive experience by placing the action in the real-world and having a more natural camera control. Using augmented reality, we will develop a game in which the objective is to rescue Kiwis, who were kidnapped by insane llamas. Players free the Kiwis by beating the insane llamas that inhabit different levels of the game. These levels are spread throughout the real-world space and seen through an iOS device, allowing players to have physical control over their point of view. A slingshot attached to the device is used to shoot vengeful Kiwis into the virtual environment and defeat the insane llamas. Parts of the virtual and real environments can be destroyed to highlight how the player can interact with both virtual and real objects. With this project, we hope to inspire novel gaming design and 3D user interaction for mobile gaming and applications.

Design Questions:
With AR we run this risk of registration errors. Is AR the best route?
  • We don’t want to alienate players just because the find the system unresponsive.
  • How much registration error is acceptable?
If there were perfect registration but no occlusion/destruction of real world objects would this be as fun/innovative/?
What does haptics afford the user? (Slingshot)
  • Is it necessary for understanding the interaction or just cool?
  • Would having to buy an additional prop deter users?
  • How can we make it fun without the prop?
On a side note, if you every need ideas for project names this website proved to be helpful for ours :D