Saturday, April 27, 2013

HCI Wireframing Assignment: Pair Programming Scheduler

These prototypes *very* loosely based on my earlier storyboards - basically I switched away from interventions during actual pairing, to think about scheduling pairing sessions.

Prototype 1 where the user indicates their availability first:




Prototype 2 where the user browsers who is available first. (more like the http://pair-with-me.herokuapp.com/ tweet aggregator)

Thursday, April 25, 2013

What I'd do with a ScreenHero API

So Screenhero rocks, but I was thinking it would be really cool if it had an API so you could scaffold the interaction between pair programmers collaborating on code.

Imagine that you've got a couple of relative novice programmers who are new to pair programming and to related concepts like TDD.  Think about being able to have suggestions pop up once they had completed certain parts of the task.  You could do this without plugging directly into ScreenHero, but what would be nice would be to ensure that one person did one portion of coding, followed by the other person doing another.

I guess what I should really do first is work out the precise flow of a scaffolded pairing session and then think more carefully about what prompts the users would need, and how to keep track of who's doing the work.  Ultimately we want to be able to have a reliable automated check that actual pair programming is happening rather than just one person doing all the work.

Perhaps having each person check their code into github separately - but that then pulls the pair apart.  I guess it raises the question of how to attribute code changes in a version repository when two people are working on the same file at the same time on one computer ...

Follow up - just watched great talk by Sam Livingston-Gray, which mentioned a pair programming protocols "Ping Pong" and some variations called "Change the Message" and "one-undermanship":


Looks like ping pong protocol goes way back.  I think this is what I'd really like to be scaffolding.

http://c2.com/cgi/wiki?PairProgrammingPingPongPattern

I wonder if the best way to scaffold would be through checkins to a Github repo - then at least you could check in retrospect how people coded ...

More HCI Bloopers

So I do appreciate the folks at Cengage Learning providing me online access to a textbook via the coursesmart system.

I was less excited about getting locked out of my account for skimming quickly through the book.  Then to unlock it there was a helpful chat window, but the multiply nested form I had to fill in to get into the chat itself was a great HCI blooper in my opinion.  It took me three or four attempts to work out which category "account locked" should go under and in the end it was none of them, so I just wasted 3 extra minutes, and eventually chose some random sub-categories:


The guy in the chat did help me out pretty promptly, but didn't seem very interested in my suggestions regarding the lack of skimming functionality or the problems with the form.  I did recommend that he direct their interface designers to the free Stanford HCI course but he didn't seem particularly interested ... :-)

Monday, April 22, 2013

Remote Pair Programming for the Masses


I am submitting this project for funding from my University.  It concerns creating an online calendar in which students from around the world can register their interest in remote pair programming.  Evidence is mounting that pair programming is the most effective way to learn how to program, and also generates higher quality code faster.  One of the key challenges is finding someone to pair with; however an appropriately structured calendar that took into account timezone differences, programming skill level etc., could create an environment where there was always someone ready to pair program with you day or night.

Project Description

Pair programming is the process whereby two programmers work together on the same coding problem.  Usually one will act as a navigator and the other as a driver with control of the keyboard.  The central idea is that there is an immediate feedback loop whereby the navigator can check the code the driver creates and the two can discuss possible alternatives.  This is similar to the kind of feedback that may come from code reviews and project retrospectives, but is on a much smaller timescale.
Pair programming is being increasingly adopted in industry although there is a general resistance based on the fear productivity will be lower given that two programmers could be working on two different tasks.  However Cockburn & Williams (2000) demonstrated the following significant benefits of pair programming:
  • many mistakes get caught as they are being typed in rather than in QA test or in the field (continuous code reviews)
  • the end defect content is statistically lower (continuous code reviews)
  • the designs are better and code length shorter (ongoing brainstorming and pair relaying)
  • the team solves problems faster (pair relaying)
  • the people learn significantly more, about the system and about software development (line-of-sight learning)
  • the project ends up with multiple people understanding each piece of the system
  • the people learn to work together and talk more often together, giving better information flow and team dynamics
  • people enjoy their work more

Cockburn & Williams go on to point out that the development cost for these benefits is not the 100% that might be expected, but is around 15%; which is quickly paid back for in shorter and less expensive testing, quality assurance, and field support.  Of particular note are the learning benefits, which have also been noted by Williams & Kessler (2003) who indicate that:
"Knowledge is constantly being passed between partners, from tool usage tips to design and programming idioms. The partners take turns being the teacher and the student. Even unspoken skills and habits cross partners."
Canfora et al (2007) found a reduction in defect rates of 15% to 50%, varying depending on programmer experience and task complexity, while a number of other studies found similar improvements (Lui, 2006; Arisholm et al., 2007; Lui et al., 2008), however meta-analyses have cautioned that this all depends on a number of factors and that you cannot expect “faster and better and cheaper” (Hannay et al., 2009).

Independently of whether pair programming is more effective than solo programming, the educational benefits seem clear.  McDowell et al (2002) and Salleh et al. (2011) both found that students’ grades and satisfaction improved with pair programming as compared to programming solo.  Furthermore McDowell et al. (2002) saw increases in retention indicating that the use of pair programming could prevent students from dropping out.

Schümmer  & Lukosch (2009) describe some of the additional difficulties faced when pair programming remotely, and there are also concerns that productivity and satisfaction can drop during novice-novice pairing is used without sufficient availability of a mentor for coaching (Stephens & Rosenberg, 2003).  All of this indicates that there is a huge pedagogical potential for remote pair programming, but that the process needs to be carefully scaffolded to ensure successful outcomes.

Specific Aims
  • Create online calendar for remote pair programming (RPP)
  • Provide support for editing, discovery and matchmaking
  • Scaffolding successful RPP sessions
  • Recording RPP sessions for analysis
  • Using results of analysis to improve the RPP experience

Significance

The significance of this work is that it is becoming clear that pair programming is the fastest and most effective way to learn how to program, and yet in general it can be very challenging to find pair programming opportunities.  A well organized calendar that allowed students from around the world to discover other students available to pair program could be invaluable in increasing student satisfaction and minimizing the amount of time required to become a confident programmer.

Work so far

We have created a shared Google calendar and sign up form that now has 30 students signed up.  Remote pair programming sessions have been recorded with at least 10 of these students and a preliminary analysis of one session has been conducted.

Bibliography
  • Arisholm, Erik; Hans Gallis, Tore Dybå, Dag I.K. Sjøberg (February 2007). "Evaluating Pair Programming with Respect to System Complexity and Programmer Expertise". IEEE Transactions on Software Engineering 33 (2): 65–86.
  • Canfora, Gerardo; Aniello Cimitile, Felix Garcia, Mario Piattini, Corrado Aaron Visaggio (2007). "Evaluating performances of pair designing in industry". The Journal of Systems and Software 80 (80): 1317–1327.
  • Cockburn, A. & Williams, L. (2000). "The Costs and Benefits of Pair Programming". Proceedings of the First International Conference on Extreme Programming and Flexible Processes in Software Engineering (XP2000).
  • Hannay, J. E.; Dybå T., Arisholm E., Sjøberg D. I. K.  (2009). "The Effectiveness of Pair Programming: A Meta-Analysis". Information and Software Technology 51 (7): 1110–1122.
  • Lui, K. M. (2006). "Pair programming productivity: Novice-novice vs. expert-expert". International Journal of Human-Computer Studies 64 (9): 915–925
  • Lui, K. M., Chan K. C. C., Nosek J. T. (2008). "The Effect of Pairs in Program Design Tasks". IEEE Transactions on Software Engineering 32 (2): 197–211.
  • McDowell, C., Werner, L., Bullock, H., & Fernald, J. (2002). The effects of pair-programming on performance in an introductory programming course. ACM SIGCSE Bulletin, 34(1), 38-42.
  • Salleh, N., Mendes, E., Grundy, J.C. (2011) Empirical Studies of Pair Programming for CS/SE Teaching in Higher Education: A Systematic Literature Review. IEEE Trans. Software Eng. 509-525.
  • Schümmer, T. & Lukosch S. (2009) "Understanding Tools and Practices for Distributed Pair Programming". Journal of Universal Computer Science 15 (16): 3101–3125.
  • Stephens M. & Rosenberg D. "Extreme Programming Refactored: The Case Against XP", Berkeley, CA: Apress, 2003.
  • Williams, Laurie; Kessler, Robert (2003). Pair Programming Illuminated. Addison-Wesley. ISBN 0-201-74576-3.





Friday, April 19, 2013

Coursera HCI StoryBoarding: Remote Pair Programming

List five existing designs (inspirations) that relate to your thinking. The relationship could be very concrete or very abstract.

1. Joe Moore's Blog: http://remotepairprogramming.com/ Joe collects lots of really useful tips on remote pair programming in his tumblr blog.

2. Android intents which allow android apps to interact with each other seemlessly. This allows re-use of components of android apps.

3. Growl notifications on OSX - which pop and then fade away. A wonderful lightweight notification system to keep you in the loop with minimum annoyance.

4. CmdShift4 screen capture on OSX - allows you to take a snapshot of some portion of the screen - not very intuitive chord key (but are they ever), but just a dream to use - select a square on the screen and image gets dropped to desktop - even better when you set the drop point to be in your dropbox public folder, since then you instantly have a publicly shareable URL to share the screen shot in text chat or email etc.

5. Automated centering guides that XCode's GUI builder (and now Google Presenetations) uses to help you drop a picture or GUI element into alignment with surrounding elements. Fantastically intuitive and easy to use - they just appear and help as you go about doing what you were already doing.

Which of the design briefs does your observation (from Assignment 1) fall under: Change, Glance, or Time? Explain what activity you observed and how it relates to the brief. Write down your Point of View.

Change. I observed programmers remote pair (triple programming) which they are doing in order to solve programming problems, but also to improve their programming skills.


Upload a comprehensive set of digital photos or scans of both of your storyboards.
Point of view: Remote Pair Programming should be as natural and easy as being in the same room with other programmers, just even more convenient.


1. Multiple cursors for different users each with colour and/or logo


2. Always front overlaid transparent chat

Coursera HCI Needfinding: Remote Pair Programming

Scott Klemmer's HCI class is great, but I think one thing that would make it even better would be if all the students design submissions were public on blogs for all to share and see (assuming student consent). I love to share, so here's my assignment 1:


Design Brief: Change

I observed groups of programmers programming together, trying to improve their programming skills and change themselves from less experienced programmers into more experienced programmers.


Describe how three people performed an activity related to the brief you chose. In particular, you should focus on a particularly interesting moment/breakdown/workaround from the observation. Upload a photograph or sketch of each person performing the described activity, with a short caption.

Three programmers worked on a particular feature from a programming project.  For reference here is the complete video: https://www.youtube.com/watch?v=ELNZTymLSaE&feature=player_embedded  I obtained permission from all programmers to share the session

Interesting moments/breakdowns/workarounds
1: microphone not working for one remote programmer and that programmer used skype text chat to stay in touch [video time 10:50, 22:03, 41:22 51:38, 1:08:48, 1:22:00]

2. One user learns from the other about a coding abbreviaton ("rails c" for "rails console") [video time 13:30]

3.  Use of reference to line numbers in text editor to help focus on particular things in the session [video time 49:56, also at 58:28]

Here are my detailed notes on the actions of the participants, M, J, and S:



3:30 working out how to add an epic in pivotal tracker

6:37 J using putty for some reason

8:44 J zooming size of screen (discussion of use of vi in the background)

10:50 S using Skype chat because his mic is not working, he can hear the other two, but they can't hear him

13:30 inspecting models from database in rails console, J types "rails c" which M had not known was contraction for "rails console"

14:55 J uses highlighting to point out link with the token for the login

15:36 J already did some test install based on link S sent? was an older version, lead to some errors but then J followed some more instructions on the devise website?

17:18 chrome gets stuck, but no big deal

19:36 M wonders where she is - navigating pivotal tracker

20:40 generating feature description? should this be coming from client

22:03 more text chat from S

26:23 M confirms the "so that" part of the feature is basically a comment

29:12 M thought that some menus were in spanish, but J says no, but they moved on

32:15 comparing tracker numbers in feature - wish we had links - there was confusion about which story they were working on

35:15 M interested in tab completion

35:45 M couldn't see the terminal at one point. J moved it down

37:52 J playing with zoom again

40:32 M suggests password field and then need for pipe symbol

41:22 Skype chat popup lets M know that S is texting to indicate 8 character restriction an be changed. Just after another popup indicates chatting on Skype in an unrelated discussion --> would be good if one could get a general heads up on the relevant chat …

44:36 M suggests changing Then to When in feature

~46:30 J asks for confirmation that they have finished everything they need in the scenario

49:28 discussing whether the user name is in the system, or only the email

49:56 M pointed out indent issues to J used line numbers to indicate where to look

50:48 M unable to hear J, asked for confirmation- he was checking again if they'd finished

51:38 S reminds them of losing password feature by text chat M tells J via audio

53:20 work out that they need to add another user story

55:00 J accidentally? clicks on Skype and then needs to minimise

56:28 M helps with spelling correction

58:28 M says to J "you have a typo on line 37" again using line numbers

59:15 M suggests a grammatical correction

1:00:20 M suggests sign in page rather than log in page for consistency

1:02:30 M has funny text message sound

1:06:51 - more tips from S in Skype

1:07:43 M corrects indentation

1:08:48 S points out cucumber doesn't require the indentation, and M acknowledges but points out it is important that it should be consistent

1:10:45 M points out some spelling mistakes

1:14:00 J has to nip off to answer phone, M pauses the screen recorder and then restarts - perhaps we lost something?

1:17:00 can hear Skype notification sounds(?) in the background

1:18:40 Other Skype users coming online notifications

1:20:29 J says he needs to finish up

1:21:00 discussion about how M can access the work that J has done on his computer

1:22:00 S points out that changes can be pushed to github repo, and there is discussion about pushing that to branch

1:23:50 saying yes in text to S, but he can hear them no?


Write a list of needs/goals/tasks inspired by what you observed

1. text chat could be spoken aloud by some automated text to speech system to allow those without microphones to participate

2. would be good to have each user have their own mouse so that they can point to things in the shared space

3. Would be good if the only relevant text chat interjected

4. Would be good if the text chat could appear without the users having to switch windows

5. Would be nice if the text editor caught some spelling mistakes in cucumber feature files

6. Would be good it important external calls could pause the screen recorder

7. Would be good if paused screen recorder had more observable "pause" status

8. Would be good if users could see more of each others status in a lightweight way, e.g. whether they are busy, distracted etc.

9. Would be good if all users could edit in parallel, e.g. fixing each others spelling mistakes

10. shared whiteboard for sketching would be good

11. bigger screen real estate for all would be good, e.g. at least two monitors for everyone and everyone able to see all of each others screens, so more like sharing an entire wall

12. would be good if the task being attempted was scaffolding in some way - could tick off when certain goals had been achieved

13. would be good if sharing and recording were always on, so that it was just like dropping into a room rather than having to set up a load of software

14. would be good to have some way to have other groups of individuals exist in the same space so that they could be called over to help on particular issues and then drift off

15. every increase in bandwidth to support higher resolution video and audio streaming would be useful for the participants

16. everyone could use help finding shared times for pair programming, coordinating the agenda, preparing to make the best use of the session



Sunday, April 14, 2013

Combining Remote Pair Programming & Massively Open Online Classes (MOOCs)

I'm getting very excited about the educational potential of remote pair programming.  We've been trying out remote pair programming on our LocalSupport project as part of the EdX Software as a Service online course.

The debate about when to use pair programming in real world projects still rages, but from my personal experience it's a huge win almost every time from an educational perspective in terms of programmers learning new languages, techniques and skills.

However one of the challenges is finding someone to pair program with.  To that end we created a shared Google Calendar that includes the times and contact details of everyone wanting to be pair program.



Programmers who want to join the calendar fill out the following form:

https://sites.google.com/site/saasellsprojects/pair-programming-form

and at the moment we then add them by hand to the calendar, as well as giving them edit permissions to allow them to change their availability over time.  Naturally in the long run we'd like to totally automate this process.

So far we've had over 10 successful remote pair programming sessions with from 2 to 5 programmers participating in each.  You can see recordings of those sessions here:

https://sites.google.com/site/saasellsprojects/projects/local-support/pair-programming

What we're also finding is a split between those relatively new to the Rails stack we are working on (mainly EdX 169.1X students) and those with more experience (those who have completed EdX 169.2X).  The latter now tend to be working on our LocalSupport non-profit project, while the former are starting to remote pair with each other to work on simple ruby and rails assignments.

I've also started to discover professionals like Joe Moore who have been remote pair programming for a number of years, and am discovering that there are a lot more options beyond the Skype and Google Hangouts we've been using so far.

What's really exciting here is the combination of the Massively Open Online Classes (MOOCs) with the up close and personal feedback of pair programming sessions.  A very powerful learning combination in my opinion.  Just navigating all the libraries in the rails stacks and getting used to git pull requests etc. is the kind of thing that pair programming really helps for.  Lots of relatively confident learners can get put off when they are stuck alone battling against these things.

If we're lucky this kind of calendar for pair programming opportunities can be embedded into MOOCs and scaled so that any time of day or night you'll always be able to find someone to pair with, and the learning opportunities will be endless ...