Sam Joseph on MOOCs for Hawaii Business Magazine

I was recently interviewed by Hawaii Business Magazine's Pavel Stankov on the subject of Massive Open Online Classes or MOOCs

Pavel: So first off, tell me about the public class that you taught online at HPU? What class was it? How long was it taught?

Sam: The class was the combined CSCI 4702 Mobile programming and MULT 4702 Mobile Design classes, and was taught over the usual 14 weeks of the HPU Spring semester this year.  It's a class that I've been teaching in one form or another at HPU and UHM for about 8 years, and focuses on the design and programming of applications for mobile devices such as tablets and smartphones.

Pavel: Who pitched the idea for that and how successful was the course? Did anybody drop out?

Sam: I designed the course myself and I pitched the idea that HPU should trial it as a public beta.  I would say the course was moderately successful :-)  I continue to get the positive feedback that I have gotten from students on this course as I have over the years.  All of the official HPU students enrolled in the course completed it, and the public format enabled some who had previously failed the course to
retake and successfully complete the course.  The majority of the non HPU students taking the course did not complete it, but I still got positive feedback from them.  I don't know that "dropping out" as a concept is particularly useful for students who are not studying for credit.  If for-credit students "drop out" it is clearly a negative event where a student has paid for support in their learning process, and for whatever reason, feels that they are not getting what they expect from a course.

When casual "MOOC" students are taking a course it is much more like they are receiving an encyclopedia.  They are gaining access to a set of materials that they can take and pick from as they please.  Since in the HPU public course trial we were not certifying their abilities I don't believe that their "drop out" rate of non-HPU students indicates the relative success or failure of the course.

Pavel: Would it be considered a MOOC, if it's not delivered through Coursera, edX, or Udacity? Was there a third party at all, or was it offered directly from HPU, just open to the public?

Sam: The first MOOCs were offered before Coursera, EdX or Udacity existed so I don't think that
who provides a course says anything about whether it should be considered a MOOC or not.
Whether we consider something a MOOC or not depends on four things, specifically it being
a course, it being available online, it being available to the public, and it being taken
by a large number of students.  Our online public course trial had around 20 students combined so I think it qualifies more as a SOOC (Small Open Online Course) than anything else :-)  I used the free and open Google Sites framework to host and deliver the course along with other free open source tools and Google App Scripts that I programmed myself.

Pavel: What class are you currently teach through edX? What is the turnout? What are the expectations and what do you hope to achieve through offering it? Have you offered a course of such a scale before? How do you feel about it?

Sam: In collaboration with UC Berkeley's Professors Dave Patterson and Armando Fox I am facilitating CS169X Software as a Service through EdX.  The current instance of the class
has over 13000 students enrolled and close to 250 teaching assistants.  The expectation
is that we can spread as widely as possible the concepts of software engineering craftsmanship.  My personal hopes are that by being closely involved in this course that my own HPU Software Engineering course will become even more valuable to the students taking it, in terms of the quality of the curricular materials they have access to, and the range of other learners they can interact with.

Sam: One of the key values of delivering a for-credit class publicly with a mix of for-credit and casual students is that the students get to mix with a much wider range of learners.  Students can take part in collaborative learning with people from all over the world, who in many cases bring fantastic industry experience with them to the class.  My involvement in this summer's EdX course is the first time that I have taken a major role in a class of this scale and I am extremely excited about it.  I see the combination of MOOC delivery systems such as EdX with personal scaffolded collaborative learning experiences such as pair programming and group projects revolutionizing the nature of the educational experience.

Pavel: What are the challenges for a MOOC instructor? What is the hardest part? The easiest part?

Sam: The challenges are having a continous flow of information with questions from students coming in constantly 24/7.  All materials have to be of exceptionally high quality. The hardest part is often just switching off for a moment to refresh yourself.  Delegation is the combined hardest and easiest part.  Given the 250 or so teaching assistants I have to control myself not to dive too deep on problems from individual students the moment they come up, leaving the teaching assistants to triage the challenges the students are facing.  The MOOC instructor must listen carefully to their teaching assistants balancing when to jump in with their expertise so as to benefit the maximum number of students possible.

Pavel:  If you were to start over, and change something in the way you approach your MOOCs, would you do so?

Sam: I am making an ongoing effort to change MOOCs so that they move away from the one size
fits all mode of learning, and focus on the individual.  I think real time interaction
with fellow MOOC students and open-ended projects of consequence to the individual
students is the key.  Certain Stanford Coursera classes have trail-blazed in this regard such as Scott Klemmer's HCI course, however I think we can go a lot lot further.

Pavel: How do you address the issues of academic honesty?

Sam: Academic honesty is an issue in a class that tries to offer credit for a largely one size fits all method of assessment.  My personal approach to academic honesty is to award credit for unique individual contributions.  To the extent possible within the contexts of HPU, Berkeley and EdX I make all my classes dependent on a students ability to offer unique individual project work.  In a class where projects are developed incrementally it quickly becomes clear if an individual is trying to attribute the work of others to themselves, even in the online context.   This is of course more complicated at large scales, but I believe we have the seeds in place to make academic honesty effectively a non-issue, and you are likely to see some very exciting developments in this regard over the next 24 months or so.

Pavel:  What is your take on the peer-grading approaches offered by some MOOC providers? Do you suppose there might be a better technology to control for plagiarizing?

Sam: I think peer-grading is an interesting approach.  I've used it in some of my classes, and I've taken MOOC classes in which it has been used.  I don't think it is yet quite in a form that delivers an ideal learning experience but it's an excellent start.  Since I teach programming and design, plagiarising is not quite the issue that is in other classes such as English and History, however it is still a concern.  I have also been taking a very interesting MOOC on cheating in online classes, and I think it's a mistake to focus on plagiarism and how to control it.  I believe the focus should be on igniting the imagination of individual students.  The key is for the instructor or teaching assistant to get to know the individual students personally, and ask them what it is they are really excited about doing.  The focus should be on providing a framework, scaffolding if you will, that enables the student to do something that they are excited about, giving them access to the tools that allow them to achieve their dreams.  I would argue that plagiarism and academic dishonesty comes largely from students not being interested in performing the academic exercises they are being set.  The solution is not to ask how to control plagiarism through technology, but what is it that interests an individual student?  Unlock the interest of the individual student and they will have no incentive to be academically dishonest; they will be truly motivated to create something of quality, and developing the necessary skills to support that.

Pavel: Do you think some classes are better suited to be taught through a MOOC than others? Which ones?

Sam: Given the form of the current MOOCs from providers such as Coursera, EdX and Udacity, I would say that computer science courses are particularly well suited to being taught through a MOOC.  One might argue that introductory courses are also well suited to MOOCs, however I am not sure I agree.  I think the being suited to a MOOC depends on who's perspective we are talking about, e.g. educator, student, institution etc.  However with the new technologies that are rolling out in terms of remote collaboration software such as Google hangouts, Multiway Skype screenshare etc. the majority of classes can be taught just as well in MOOC format, if not better than at a physical institution.  Although a clear exception would be those courses that are attempting to train or instruct in the use of expensive specialist equipment that is not available to the individual at their home.

Pavel:  Under what circumstances should colleges and universities award academic credit for third-party MOOC providers?

Sam: At the moment I think it's not in colleges and universities interest to award academic credit for third-party MOOC providers unless assessment has been verified through proctored examinations.

Pavel: How do you see the development of MOOCs in the observable future?

Sam: Difficult to say although my personal hope and plan is that MOOCs will become a lot more accessible and personal with MOOCs supporting students to talk in real time with their peers and instructors.

Pavel: Is there anything in particular about Hawaii that makes our location different when it comes to online higher education?

Sam: Hawaii of course has a large military presence; and military students really must have online education due to the nature of their work.  Of course Hawaii is also geographically isolated meaning that the range of educational choices is not what it is in other areas, making online education particularly valuable to Hawaii residents.

Pavel: How do you react to comments that MOOC are undermining higher education? For instance, some people are inclined to say that if awarded with academic credit, they would have no incentive to physically go to school when they can take everything online for free or a symbolic fee? Do you think this is a valid argument? Do you think this is a problem for faculty?

Sam: I guess the argument is that as MOOCs start awarding academic credit then many students might not attend a bricks and mortar "academy", thus undermining higher education institutions?  I think it's entirely possible that MOOCs may undermine higher education institutions that are not offering the highest quality of courses and value for money to their students.  I don't think that anything that itself promotes education can undermine education unless one posits that students receive some particular benefit from attending a physical institution that they cannot receive online.  Individuals have been taking distance and online courses around the world for many many years.  Here in the UK the Open University has been delivering higher education without physical institutions for some 30 odd years, and the UK still has a very strong set of higher education institutions in the physical domain.  I think it is an open question as to the value that students receive from attending a physical institution, and the most important thing in a free market is that of choice.  Students should have a free choice as to where they look for support to achieve their learning goals, and should not be paying over the odds for the support they receive.  I think the only challenge that MOOCs and other educational technology developments present for faculty is for those faculty who are not delivering the highest quality educational experience possible, and for those institutions who are charging over the odds.

Pavel: Finally, is there anything that you want to add, or a question that i should have asked you?

A general comment would be that globally I think we have education back to front.  Rather than asking what skills we should be teaching students and what subjects they should be learning, we should be asking our students what excites them, what they want to learn and what they want to build.

Scheduling Remote Pair Programming Sessions

So maybe it's just me, but it feels like there has been a recent explosion in interest in remote pair programming. Maybe it's being going on for ages, but I recently discovered great resources such as Joe Moore's http://remotepairprogramming.com/ and http://www.pairprogramwith.me/

I've now added the PairWithMe badge to all my projects on Github, and I follow the #pairwithme hashtag on twitter, which is exposing me to things like this remote pair programming meetup group:

http://www.meetup.com/remotepairprogrammers/

@batarski and @eee_c were saying on twitter that the #pairwithme website should coordinate and maybe host the sessions.  I gave a lame tweet reply that that would be cool, and that some other sites were doing that, but didn't really have space to list the, so here they are:

• http://pairwith.me/  Alpha - very basic
• http://rubypair.com/ Gamma? - looks cool
• http://tansaku.github.io/PairProgrammingScheduler/ Alpha - hardly functional - this is my attempt so far :-)
• https://www.virtualpairprogrammers.com/ Gamma? Paid site - training courses
• https://github.com/samnang/pair-with-me Alpha?  Not functional any more?

And there was another one I signed up with recently, but I lost my Chrome history in a recent crash, so can't find it at the moment :-(

However although I've made contact with a few people through these (mainly just on twitter) I think I've only got a couple of remote pair programming sessions going from them.  Far more intensive has been the now 30+ and counting hours I've been remote pair programming on our EdX SaaS LocalSupport project which we are mainly co-ordinating through Skype chat rooms.  Here's a playlist of those videos

http://www.youtube.com/playlist?list=PLjbL0BCR04Q1-x5p5L5V11Vm58VlTgPzw

I'm looking forward to seeing more solutions to the challenge of remote pair programming scheduling.

HCI Wireframing Assignment: Pair Programming Scheduler

These prototypes *very* loosely based on my earlier storyboards - basically I switched away from interventions during actual pairing, to think about scheduling pairing sessions.

Prototype 1 where the user indicates their availability first:




Prototype 2 where the user browsers who is available first. (more like the http://pair-with-me.herokuapp.com/ tweet aggregator)

What I'd do with a ScreenHero API

So Screenhero rocks, but I was thinking it would be really cool if it had an API so you could scaffold the interaction between pair programmers collaborating on code.

Imagine that you've got a couple of relative novice programmers who are new to pair programming and to related concepts like TDD.  Think about being able to have suggestions pop up once they had completed certain parts of the task.  You could do this without plugging directly into ScreenHero, but what would be nice would be to ensure that one person did one portion of coding, followed by the other person doing another.

I guess what I should really do first is work out the precise flow of a scaffolded pairing session and then think more carefully about what prompts the users would need, and how to keep track of who's doing the work.  Ultimately we want to be able to have a reliable automated check that actual pair programming is happening rather than just one person doing all the work.

Perhaps having each person check their code into github separately - but that then pulls the pair apart.  I guess it raises the question of how to attribute code changes in a version repository when two people are working on the same file at the same time on one computer ...

Follow up - just watched great talk by Sam Livingston-Gray, which mentioned a pair programming protocols "Ping Pong" and some variations called "Change the Message" and "one-undermanship":

Looks like ping pong protocol goes way back.  I think this is what I'd really like to be scaffolding.

http://c2.com/cgi/wiki?PairProgrammingPingPongPattern

I wonder if the best way to scaffold would be through checkins to a Github repo - then at least you could check in retrospect how people coded ...

More HCI Bloopers

So I do appreciate the folks at Cengage Learning providing me online access to a textbook via the coursesmart system.

I was less excited about getting locked out of my account for skimming quickly through the book.  Then to unlock it there was a helpful chat window, but the multiply nested form I had to fill in to get into the chat itself was a great HCI blooper in my opinion.  It took me three or four attempts to work out which category "account locked" should go under and in the end it was none of them, so I just wasted 3 extra minutes, and eventually chose some random sub-categories:

The guy in the chat did help me out pretty promptly, but didn't seem very interested in my suggestions regarding the lack of skimming functionality or the problems with the form.  I did recommend that he direct their interface designers to the free Stanford HCI course but he didn't seem particularly interested ... :-)

Remote Pair Programming for the Masses

I am submitting this project for funding from my University.  It concerns creating an online calendar in which students from around the world can register their interest in remote pair programming.  Evidence is mounting that pair programming is the most effective way to learn how to program, and also generates higher quality code faster.  One of the key challenges is finding someone to pair with; however an appropriately structured calendar that took into account timezone differences, programming skill level etc., could create an environment where there was always someone ready to pair program with you day or night.

Project Description

Pair programming is the process whereby two programmers work together on the same coding problem.  Usually one will act as a navigator and the other as a driver with control of the keyboard.  The central idea is that there is an immediate feedback loop whereby the navigator can check the code the driver creates and the two can discuss possible alternatives.  This is similar to the kind of feedback that may come from code reviews and project retrospectives, but is on a much smaller timescale.

Pair programming is being increasingly adopted in industry although there is a general resistance based on the fear productivity will be lower given that two programmers could be working on two different tasks.  However Cockburn & Williams (2000) demonstrated the following significant benefits of pair programming:

• many mistakes get caught as they are being typed in rather than in QA test or in the field (continuous code reviews)
• the end defect content is statistically lower (continuous code reviews)
• the designs are better and code length shorter (ongoing brainstorming and pair relaying)
• the team solves problems faster (pair relaying)
• the people learn significantly more, about the system and about software development (line-of-sight learning)
• the project ends up with multiple people understanding each piece of the system
• the people learn to work together and talk more often together, giving better information flow and team dynamics
• people enjoy their work more

Cockburn & Williams go on to point out that the development cost for these benefits is not the 100% that might be expected, but is around 15%; which is quickly paid back for in shorter and less expensive testing, quality assurance, and field support.  Of particular note are the learning benefits, which have also been noted by Williams & Kessler (2003) who indicate that:

"Knowledge is constantly being passed between partners, from tool usage tips to design and programming idioms. The partners take turns being the teacher and the student. Even unspoken skills and habits cross partners."

Canfora et al (2007) found a reduction in defect rates of 15% to 50%, varying depending on programmer experience and task complexity, while a number of other studies found similar improvements (Lui, 2006; Arisholm et al., 2007; Lui et al., 2008), however meta-analyses have cautioned that this all depends on a number of factors and that you cannot expect “faster and better and cheaper” (Hannay et al., 2009).

Independently of whether pair programming is more effective than solo programming, the educational benefits seem clear.  McDowell et al (2002) and Salleh et al. (2011) both found that students’ grades and satisfaction improved with pair programming as compared to programming solo.  Furthermore McDowell et al. (2002) saw increases in retention indicating that the use of pair programming could prevent students from dropping out.

Schümmer  & Lukosch (2009) describe some of the additional difficulties faced when pair programming remotely, and there are also concerns that productivity and satisfaction can drop during novice-novice pairing is used without sufficient availability of a mentor for coaching (Stephens & Rosenberg, 2003).  All of this indicates that there is a huge pedagogical potential for remote pair programming, but that the process needs to be carefully scaffolded to ensure successful outcomes.

Specific Aims

• Create online calendar for remote pair programming (RPP)
• Provide support for editing, discovery and matchmaking
• Scaffolding successful RPP sessions
• Recording RPP sessions for analysis
• Using results of analysis to improve the RPP experience

Significance

The significance of this work is that it is becoming clear that pair programming is the fastest and most effective way to learn how to program, and yet in general it can be very challenging to find pair programming opportunities.  A well organized calendar that allowed students from around the world to discover other students available to pair program could be invaluable in increasing student satisfaction and minimizing the amount of time required to become a confident programmer.

Work so far

We have created a shared Google calendar and sign up form that now has 30 students signed up.  Remote pair programming sessions have been recorded with at least 10 of these students and a preliminary analysis of one session has been conducted.

Bibliography

• Arisholm, Erik; Hans Gallis, Tore Dybå, Dag I.K. Sjøberg (February 2007). "Evaluating Pair Programming with Respect to System Complexity and Programmer Expertise". IEEE Transactions on Software Engineering 33 (2): 65–86.
• Canfora, Gerardo; Aniello Cimitile, Felix Garcia, Mario Piattini, Corrado Aaron Visaggio (2007). "Evaluating performances of pair designing in industry". The Journal of Systems and Software 80 (80): 1317–1327.
• Cockburn, A. & Williams, L. (2000). "The Costs and Benefits of Pair Programming". Proceedings of the First International Conference on Extreme Programming and Flexible Processes in Software Engineering (XP2000).
• Hannay, J. E.; Dybå T., Arisholm E., Sjøberg D. I. K.  (2009). "The Effectiveness of Pair Programming: A Meta-Analysis". Information and Software Technology 51 (7): 1110–1122.
• Lui, K. M. (2006). "Pair programming productivity: Novice-novice vs. expert-expert". International Journal of Human-Computer Studies 64 (9): 915–925
• Lui, K. M., Chan K. C. C., Nosek J. T. (2008). "The Effect of Pairs in Program Design Tasks". IEEE Transactions on Software Engineering 32 (2): 197–211.
• McDowell, C., Werner, L., Bullock, H., & Fernald, J. (2002). The effects of pair-programming on performance in an introductory programming course. ACM SIGCSE Bulletin, 34(1), 38-42.
• Salleh, N., Mendes, E., Grundy, J.C. (2011) Empirical Studies of Pair Programming for CS/SE Teaching in Higher Education: A Systematic Literature Review. IEEE Trans. Software Eng. 509-525.
• Schümmer, T. & Lukosch S. (2009) "Understanding Tools and Practices for Distributed Pair Programming". Journal of Universal Computer Science 15 (16): 3101–3125.
• Stephens M. & Rosenberg D. "Extreme Programming Refactored: The Case Against XP", Berkeley, CA: Apress, 2003.
• Williams, Laurie; Kessler, Robert (2003). Pair Programming Illuminated. Addison-Wesley. ISBN 0-201-74576-3.

Coursera HCI StoryBoarding: Remote Pair Programming

List five existing designs (inspirations) that relate to your thinking. The relationship could be very concrete or very abstract.

1. Joe Moore's Blog: http://remotepairprogramming.com/ Joe collects lots of really useful tips on remote pair programming in his tumblr blog.

2. Android intents which allow android apps to interact with each other seemlessly. This allows re-use of components of android apps.

3. Growl notifications on OSX - which pop and then fade away. A wonderful lightweight notification system to keep you in the loop with minimum annoyance.

4. CmdShift4 screen capture on OSX - allows you to take a snapshot of some portion of the screen - not very intuitive chord key (but are they ever), but just a dream to use - select a square on the screen and image gets dropped to desktop - even better when you set the drop point to be in your dropbox public folder, since then you instantly have a publicly shareable URL to share the screen shot in text chat or email etc.

5. Automated centering guides that XCode's GUI builder (and now Google Presenetations) uses to help you drop a picture or GUI element into alignment with surrounding elements. Fantastically intuitive and easy to use - they just appear and help as you go about doing what you were already doing.

Which of the design briefs does your observation (from Assignment 1) fall under: Change, Glance, or Time? Explain what activity you observed and how it relates to the brief. Write down your Point of View.

Change. I observed programmers remote pair (triple programming) which they are doing in order to solve programming problems, but also to improve their programming skills.


Upload a comprehensive set of digital photos or scans of both of your storyboards.
Point of view: Remote Pair Programming should be as natural and easy as being in the same room with other programmers, just even more convenient.

1. Multiple cursors for different users each with colour and/or logo

2. Always front overlaid transparent chat

Coursera HCI Needfinding: Remote Pair Programming

Scott Klemmer's HCI class is great, but I think one thing that would make it even better would be if all the students design submissions were public on blogs for all to share and see (assuming student consent). I love to share, so here's my assignment 1:


Design Brief: Change

I observed groups of programmers programming together, trying to improve their programming skills and change themselves from less experienced programmers into more experienced programmers.


Describe how three people performed an activity related to the brief you chose. In particular, you should focus on a particularly interesting moment/breakdown/workaround from the observation. Upload a photograph or sketch of each person performing the described activity, with a short caption.

Three programmers worked on a particular feature from a programming project.  For reference here is the complete video: https://www.youtube.com/watch?v=ELNZTymLSaE&feature=player_embedded  I obtained permission from all programmers to share the session

Interesting moments/breakdowns/workarounds

1: microphone not working for one remote programmer and that programmer used skype text chat to stay in touch [video time 10:50, 22:03, 41:22 51:38, 1:08:48, 1:22:00]

2. One user learns from the other about a coding abbreviaton ("rails c" for "rails console") [video time 13:30]

3.  Use of reference to line numbers in text editor to help focus on particular things in the session [video time 49:56, also at 58:28]

Here are my detailed notes on the actions of the participants, M, J, and S:

3:30 working out how to add an epic in pivotal tracker

6:37 J using putty for some reason

8:44 J zooming size of screen (discussion of use of vi in the background)

10:50 S using Skype chat because his mic is not working, he can hear the other two, but they can't hear him

13:30 inspecting models from database in rails console, J types "rails c" which M had not known was contraction for "rails console"

14:55 J uses highlighting to point out link with the token for the login

15:36 J already did some test install based on link S sent? was an older version, lead to some errors but then J followed some more instructions on the devise website?

17:18 chrome gets stuck, but no big deal

19:36 M wonders where she is - navigating pivotal tracker

20:40 generating feature description? should this be coming from client

22:03 more text chat from S

26:23 M confirms the "so that" part of the feature is basically a comment

29:12 M thought that some menus were in spanish, but J says no, but they moved on

32:15 comparing tracker numbers in feature - wish we had links - there was confusion about which story they were working on

35:15 M interested in tab completion

35:45 M couldn't see the terminal at one point. J moved it down

37:52 J playing with zoom again

40:32 M suggests password field and then need for pipe symbol

41:22 Skype chat popup lets M know that S is texting to indicate 8 character restriction an be changed. Just after another popup indicates chatting on Skype in an unrelated discussion --> would be good if one could get a general heads up on the relevant chat …

44:36 M suggests changing Then to When in feature

~46:30 J asks for confirmation that they have finished everything they need in the scenario

49:28 discussing whether the user name is in the system, or only the email

49:56 M pointed out indent issues to J used line numbers to indicate where to look

50:48 M unable to hear J, asked for confirmation- he was checking again if they'd finished

51:38 S reminds them of losing password feature by text chat M tells J via audio

53:20 work out that they need to add another user story

55:00 J accidentally? clicks on Skype and then needs to minimise

56:28 M helps with spelling correction

58:28 M says to J "you have a typo on line 37" again using line numbers

59:15 M suggests a grammatical correction

1:00:20 M suggests sign in page rather than log in page for consistency

1:02:30 M has funny text message sound

1:06:51 - more tips from S in Skype

1:07:43 M corrects indentation

1:08:48 S points out cucumber doesn't require the indentation, and M acknowledges but points out it is important that it should be consistent

1:10:45 M points out some spelling mistakes

1:14:00 J has to nip off to answer phone, M pauses the screen recorder and then restarts - perhaps we lost something?

1:17:00 can hear Skype notification sounds(?) in the background

1:18:40 Other Skype users coming online notifications

1:20:29 J says he needs to finish up

1:21:00 discussion about how M can access the work that J has done on his computer

1:22:00 S points out that changes can be pushed to github repo, and there is discussion about pushing that to branch

1:23:50 saying yes in text to S, but he can hear them no?


Write a list of needs/goals/tasks inspired by what you observed

1. text chat could be spoken aloud by some automated text to speech system to allow those without microphones to participate

2. would be good to have each user have their own mouse so that they can point to things in the shared space

3. Would be good if the only relevant text chat interjected

4. Would be good if the text chat could appear without the users having to switch windows

5. Would be nice if the text editor caught some spelling mistakes in cucumber feature files

6. Would be good it important external calls could pause the screen recorder

7. Would be good if paused screen recorder had more observable "pause" status

8. Would be good if users could see more of each others status in a lightweight way, e.g. whether they are busy, distracted etc.

9. Would be good if all users could edit in parallel, e.g. fixing each others spelling mistakes

10. shared whiteboard for sketching would be good

11. bigger screen real estate for all would be good, e.g. at least two monitors for everyone and everyone able to see all of each others screens, so more like sharing an entire wall

12. would be good if the task being attempted was scaffolding in some way - could tick off when certain goals had been achieved

13. would be good if sharing and recording were always on, so that it was just like dropping into a room rather than having to set up a load of software

14. would be good to have some way to have other groups of individuals exist in the same space so that they could be called over to help on particular issues and then drift off

15. every increase in bandwidth to support higher resolution video and audio streaming would be useful for the participants

16. everyone could use help finding shared times for pair programming, coordinating the agenda, preparing to make the best use of the session

Combining Remote Pair Programming & Massively Open Online Classes (MOOCs)

I'm getting very excited about the educational potential of remote pair programming.  We've been trying out remote pair programming on our LocalSupport project as part of the EdX Software as a Service online course.

The debate about when to use pair programming in real world projects still rages, but from my personal experience it's a huge win almost every time from an educational perspective in terms of programmers learning new languages, techniques and skills.

However one of the challenges is finding someone to pair program with.  To that end we created a shared Google Calendar that includes the times and contact details of everyone wanting to be pair program.



Programmers who want to join the calendar fill out the following form:

https://sites.google.com/site/saasellsprojects/pair-programming-form

and at the moment we then add them by hand to the calendar, as well as giving them edit permissions to allow them to change their availability over time.  Naturally in the long run we'd like to totally automate this process.

So far we've had over 10 successful remote pair programming sessions with from 2 to 5 programmers participating in each.  You can see recordings of those sessions here:

https://sites.google.com/site/saasellsprojects/projects/local-support/pair-programming

What we're also finding is a split between those relatively new to the Rails stack we are working on (mainly EdX 169.1X students) and those with more experience (those who have completed EdX 169.2X).  The latter now tend to be working on our LocalSupport non-profit project, while the former are starting to remote pair with each other to work on simple ruby and rails assignments.

I've also started to discover professionals like Joe Moore who have been remote pair programming for a number of years, and am discovering that there are a lot more options beyond the Skype and Google Hangouts we've been using so far.

What's really exciting here is the combination of the Massively Open Online Classes (MOOCs) with the up close and personal feedback of pair programming sessions.  A very powerful learning combination in my opinion.  Just navigating all the libraries in the rails stacks and getting used to git pull requests etc. is the kind of thing that pair programming really helps for.  Lots of relatively confident learners can get put off when they are stuck alone battling against these things.

If we're lucky this kind of calendar for pair programming opportunities can be embedded into MOOCs and scaled so that any time of day or night you'll always be able to find someone to pair with, and the learning opportunities will be endless ...

Benefit of Quizzes for Learning

In a recent discussion I was pointed to this article on the benefits of testing as evidence that quizzes have pedagogical value. Given that memory researchers have been getting results that testing students improved their retention for many years I was a little surprised that the NYT was presenting these results as quite so novel.  For example Dempster (1996) provides multiple citations of this effect going back as early as the 1920s. I guess that's journalism for you :-)  However the study being reported showed that simple testing beat drawing concept maps which is very interesting, although I think the results might be reversed if the students were practising generating concept maps from memory in the concept map condition.

This is an area that I have researched in some detail, particularly as regards vocabulary learning:

Joseph S.R.H., Watanabe Y., Shiung Y.-J., Choi B. & Robbins C. (2009) Key Aspects of Computer Assisted Vocabulary Learning (CAVL): Combined Effects of Media, Sequencing and Task Type. Research and Practice in Technology Enhanced Learning.
4(2) 1-36.

I certainly don't doubt that retrieval practise promotes retention, or more simple that taking tests helps you remember things.  I think the problem with the study mentioned in the NYT and those referred to by Dempster is that what they are really showing is that taking a test is great preperation for doing well on tests.  If the objective is to have students pass tests then yes, let them do lots of tests.

A more important question is what are the target skills that we are hoping that students acquire?  For example in my programming classes I am hoping that my students will learn to program.  So should I be setting them written tests, or should I be setting them programming assignments?

In my HCI classes I want my students to learn design skills, to produce good designs, so should I set them written tests on the subject of design, or should I have them do design assignments?

Scott Klemmer's HCI classes have lots of great assignments in which students practise design skills, as well as standalone and media integrated quizzes.  I don't think there is any doubt as to value of the assignments, but are quizzes the best way to get the students to the point that they can practise the target skills?  Simple quizzes built into the lecture material seem relatively benign, and these are common place throughout the current slew of MOOC courses, and are arguably good at keeping student focus on the short videos provided.  I have also been impressed with the use of quizzes with voting and discussion as seen in the Berkeley Software Engineering classes.  However for students in MOOC land they are watching the videos in a variety of settings and I wonder if the short quizzes are really worth the effort.

Furthermore I think longer harder quizzes such as those in the Berkeley Software Engineering course (both face to face and online) can be rather intimidating for many students.  And even for confident students, quizzes can be fundamentally a boring activity.  Of course quizzes give instructors insight into student progress, but they are also potentially demotivating for students and one might reasonably ask if they have any effect on the ability of the students to perform the relevant target skills?

Given that we are considering courses that will provide at least some material in text and video and other media, the key question to me is whether quizzes integrated into the media are directly beneficial in terms of subsequent performance on target skills?

To operationalize all this we might try to device a study to test if is more valuable for a student to be spending it absorbing material passively and then taking a short quiz, or absorbing more material passively, or absorbing it passively and than doing an active assignment, or perhaps just doing an active assignment with the whole of that time period?  What might our experimental conditions look like?  These perhaps:

• 10 mins passive material, 5 mins on quiz
• 15 mins on passive material
• 10 mins passive material, 5 mins on active relevant assignment
• 15 mins on active relevant assignment

And by a relevant assignment, I mean ideally something that is relevant to that individual student, such as a project that they are actually motivated to work on.  It seems like there is a great opportunity for AB testing in the current MOOCs - give a randomly selected half of the students no quizzes during the lectures, and the other half the integrated quizzes, and compare the quality of outcomes.  Or three conditions perhaps:

A) No Quizzes
B) Simple Quizzes
C) Harder Quizzes

Maybe given some chunk of time, say 15 minutes, it's unrealistic to have every student work on an active relevant assignment - maybe given 15 minutes at the end of the day, all people can manage is some passive material and then a quick quiz to help them absorb it?

On a more ethereal note I start to wonder if it is ever very effective to provide information to a student if they haven't expressed a need for it, or at least an interest in it.  For most effective learning perhaps we need to manuever the student (or perhaps ourselves?) into a position where they ask a question of their own violition and then respond to them with an answer and perhaps a further question?



Dempster, F. N. (1996). Distributing and managing the conditions of encoding and practice. In
R. Bjork, & E. Bjork (Eds.), Memory (pp. 317–344). San Diego, CA: Academic Press

HyperLocal News in JustInMind

Building on my earlier Balsamiq prototype here's another version of the HyperLocalNews app using the JustInMind framework.   This gives a much higher fidelity prototype which to the untrained eye might look more like the real thing.  The danger here is that a client might think you've already built the entire app :-)

Try clicking on the map markers:


Many thanks to creative commons sharing folk for the images to make the news stories more interesting:

http://www.flickr.com/photos/7700821@N06/476498157/
http://www.flickr.com/photos/danshouse/137241182/
http://www.flickr.com/photos/us-pacific-command/7658198802/

Note that this is different from the Balsamiq exported PDF in that it's all HTML, and there's no accidental scrolling up and down.  Getting a good look and feel of course relies on one knowing where to position things as they would appear on the appropriate smartphone.

Fest Swing Testing in Netbeans IDE 7.3

I wanted to see if I could get the Fest Swing GUI testing framework running in NetBeans IDE (7.3).   The existing plugin was designed for 6.1; and some of the instructional images were missing.  Once I'd worked out how to install Netbeans IDE plugins I found it did work with 7.3.  Here's my version getting the sample code running in NetBeans IDE 7.3 beta 2

1. Create a File --> New Project

2. Choose Samples --> Java --> Anagram Game

















3. Right click "Test Libraries" and choose Fest and JUnit 4.10

4.  Create a File --> New File, and select Fest Test from Swing GUI forms:

5. Give it an appropriate name and put it in the com.toy.anagrams.ui package

6. Add these three lines after the initComponents() method call in the constructor in Anagrams.java

        guessedWord.setName("anagramField");
        guessButton.setName("guessButton");
        feedbackLabel.setName("resultLabel");

7.  Uncomment line 26 in the fest file you created:

8. Execute the fest file as a test, and watch the Anagram game be have text filled in automatically

Interactive HyperLocal News Mockup

Balsamiq is a great tool for quickly mocking up an interface, and you can even add interactivity, and my embedding a PDF export, you can have an interactive mockup running in your browser.

Try clicking on the map pins in the PDF below:

Great 404 page design from balsamiq

2013 Mac Book Pro Reset button is pressed a bit too easily ...

So in what may hopefully be an ongoing series of design bloopers here's something from my 2013 Mac Book Pro.  The reset button was previously completely separate from the keyboard, but has now been placed just above the delete key, so that I accidentally hit it a few times a week.

It does have a confirmation dialog to prevent me immediately shutting down my computer when I'm typing, but it seems like a bit of a hack.  I guess there must be some strong technical reason for having changed this around, but I'm not seeing much in the way of improvement in terms of user experience here ...

This seems to fit well with some of Nielsen's usability heuristics.  It does well in terms of "Help users recognize, diagnose, and recover from errors", but perhaps not so well in terms of "Error Prevention"

Error prevention
Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.

They've got the confirmation action, but better to design to avoid error occurring in the first place, and Raskin hates confirmation dialogs ... It's particularly odd since the previous mac book pros were designed such that this was never an issue - the reset button was out of reach in a silvered portion of the body of the MBP in the top right, and was set into the frame so that it was difficult to press accidentally ...

How NOT to do email notifications

So in an effort to collect user interface (UI) bloopers, here's a pet peeve of mine.  Google Plus notifications.  This is how they look in the subject line view of Thunderbird.  Look at this awful replication of information.  The From field is telling me who it is from and that it's via Google+.  The last thing I want is the same information in the subject line.

I don't think anyone could argue (in the context of this interface) that the subject line should be anything other than something about the contents of the post - the first sentence perhaps?  The title of an associated URL that was being shared.  Anything to let me know something more about what I might discover when I click through.

Perhaps the problem is that the Google plus notification team hasn't optimized for Thunderbird?  Fair enough - I can see Thunderbird users as a bit of a niche group, perhaps they are thinking in terms of the gmail interface:

It would appear not - exactly the same problem in gmail.  Although to be fair when I haven't searched for Google+ the faded out text on the right hand side does have a small amount of text from the actual post.  Although if we are on search in gmail, why isn't there a sensible way for me to get back to the full inbox other than clicking inbox after a search - if I delete the search term in the box and press search shouldn't I go back to my full inbox?

Anyway maybe these Google plus notifications are designed for viewing on a mobile device?  This is what it looks like on my Nexus:

Come on Google - this is really simple stuff no?  I know I couldn't pass your complex engineering exam, but when are you going to hire me to sort out these UI issues for you ...

Harrow Local Support StakeHolders

Image by Jurgen Appelo thru CC BY 2.0

One of my continuing posts related to the Systems Analysis / Software Engineering class I'm teaching this semester.  Here's a list of stakeholders for a "Harrow Local Support" website/directory concept I've been working on with Voluntary Action Harrow (VAH) an umbrella group for local charities and non-profits.

Harrow residents

• Looking for support
• Looking for volunteering opportunities

Harrow Charities

• Trustees - who oversee the actions of a charities
• Administrators - who run the charity
• Staff - who work for the charity
• Volunteers - who devote their time/resources for free

Harrow Non-profits (groups without charitable status, and also ???? unclear if there is a legal definition)

• Administrators
• Staff
• Volunteers

Voluntary Action Harrow (VAH)

• Administrator
• Staff
• Volunteers

Other organizations

• Harrow Businesses
• Harrow Council

Stakeholders sounds a bit management faddish, but I guess it's a reasonable term that distinguishes itself from alternatives such as "people" and "users", which can lead one down the path of imagining a queue of people lining up to use whatever system one happens to create.  Realizing that everyone is busy playing parts and roles in all sorts of other groups is important, and stakeholders captures the idea that individuals (and perhaps organizations) will have varying degrees of "stake" in whatever project you are developing.

Eternal Independent Learning (using blogs and peer review)

Image by Wesley Fryer thru CC BY-SA

I was very impressed with the Peer Review framework Scott Klemmer had in his Coursera HCI classes last year.  I'm implementing simpler versions of that in my own online classes, but with all the student work submitted and reviewed through student blogs (I recommend, but don't require, using Google's http://blogger.com ).

https://sites.google.com/site/hpumobilespring2013/blogs

I really like the blogging in the wild approach since it ensures that when someone is commenting on their peer's work, they can (at least in principle) see the full history of the project being worked on by their peer, and if they are interested in can keep coming back to it, offer to collaborate etc.

I think this at least partially addresses some issues I encountered in the Coursera peer review system such as not being able to see the broader project context of peer work, and not being able to follow the development of peer projects, however it raises a number of new issues:

1) Are students happy with blogging in "public"?

2) How to scaffold the peer review process in public blogs

Addressing point 1) I explain to all my students that they can set the blogger to:

a) Full Public - appears in Google search results

b) Accessible to all by link

c) Private

And I explain that even with option b that in principle anyone can come through my public class site and find their blog unless it is explicitly set to private; but I encourage everyone to take the b) option.  I'm a bit of a sharing nut :-)  And I hope all my students can turn their projects into part of their professional portfolio.

Regarding point 2) I find myself equivocating on how much structure should be imposed on students both for assignments and peer review.  Too strong a structure and some student creativity is inhibited, too loose a structure and perhaps some students are left unsure how to proceed; not to mention the added concerns of encourage positive helpful feedback during pair review.

Any which way it makes me imagine an "Eternal Independent Learning" system in which individuals would be working on projects, posting them to their blogs and receiving feedback from each other.  Clearly this is what actually happens in the real blogosphere itself to some extent :-) But I'm imagining a little bit of scaffolding using the blogger API could create a sort of queue of people waiting for feedback on their projects and do something like http://www.fivesecondtest.com/ except on a larger scale so that arbitrary project components (rather than just landing pages) could gain reciprocal feedback, while also allowing individuals to establish mutually supportative relationships with each other.

One could even go further to imagine a StackOverflow/Quora type points system that gave individuals a score based on their contributions to other blogs, and the positive feedback they attract on their own blogs, peer's setting each other challenges etc. 

Comedy Faqbot Project

Image from Public Domain

So this is the final of three project brainstorming posts related to the Systems Analysis / Software Engineering class I'm teaching this semester.  I've been wrestling with a chatbot that can answer some useful questions about online classes (and tell a few jokes) for some time.  This project is also bundled up with my student work assignment tracker.

Problem Description:

Courses that are run year after year build up a set of previously answered questions that it would be great if current students and alumni (and instructors) had access to.  One form is just to display these in static text, but another form would be to provide a chatbot interface that could participate in IM discussions.  Boredom is the mind killer when it comes to learning, a little bit of quality comedy might help?

System Capabilities:

• Answer simple questions relating to an online course
• Provide students with information about which assignments they have completed
• Tell the occasional joke, riffing off what the student is asking

Benefits

• Handle much larger number of students for a single instructor
• Provide an alternative form of access to a large repository of information
• Provide some occasional light relief

Prototype open sourced here:

https://github.com/tansaku/faqbot

HyperLocal News Project

So this is the second of three project brainstorming posts related to the Systems Analysis / Software Engineering class I'm teaching this semester.  I've been pondering about a hyperlocal news system for a number of years: http://linklens.blogspot.co.uk/search?q=maps and I guess I should be contacting a local news organization like the Harrow Observer to see if I can't get some geocoded local news data from them

Problem Description:

Local residents may not know what is going on around them.  They may have a general idea from newspapers, tv news etc., but there is something about a map overlay to really highlight where things are happening nearby

System Capabilities:

• Provide a list of news items on a map of the local area
• Support discussion of those news items by local residents

Benefits

• Increased community engagement in local stories
• Promote local news outlets

Harrow Local Support Project

So this is the first of three project brainstorming posts related to the Systems Analysis / Software Engineering class I'm teaching this semester.  I've been volunteering with Voluntary Action Harrow for a few months now, and we are working on an idea for a website/directory that is tentatively titled "Harrow Local Support".  Here's a description in the "System Vision Format" from Satzinger et al's "System Design and Analysis in Changing World"

Problem Description:

Local residents may have trouble finding support services when loved ones fall ill, pass away, lose work or encounter any kind of challenge that is not affordably addressed by for-profit organizations.  Charities and non-profits providing support services in the local area have difficulty making local residents aware of their activities or even discovering the existence of each other.  Voluntary Action Harrow would like to have some sort of directory service that would allow both local residents and local non-profits to easily find one another

System Capabilities:

• A directory of local support services that is easily searchable and discoverable by non-technical users
• Ability for local support services to update their contact details 
• Allow Voluntary Action Harrow to promote the services they offer to non-profits, and the events they organize
• Provide a controllable web presence for small charities that can't easily create their own websites.

Benefits

• Increase VAH profile
• Support local charities, community and residents

Prototype site here:

https://sites.google.com/site/hlsbeta123123123123/

Chrome tough on http in https - how to get round it

Some students and myself have noticed our dropbox hosted html files failing in Chrome when trying to load bits of JavaScript recently ("The page at XYZ ran insecure content from ABC").  The problem seems to be that your standard dropbox public URL link defaults to secure http, i.e. https:// ... whereas the file that you might have dropped in your dropbox Public folder might be trying to pull in some CSS and JavaScript files over simple unsecure http.  On Firefox no problem, but on Chrome this is now flagged as insecure, the files are not pulled in and you get a lack of CSS and JavaScript errors due to libraries not being loaded.

Here's an example of this failure:

https://dl.dropbox.com/u/13196858/Spr2013Mobile/jQueryMobile/index.html

The quick fix (if you can't switch all the CSS and JavaScript libraries to https), is to remove the "s" from the http, and download insecurely from dropbox. Not that this is a big deal for prototyping - it isn't, but confusing since the right click grab of the public dropbox url defaults to https ...

http://dl.dropbox.com/u/13196858/Spr2013Mobile/jQueryMobile/index.html

Or alternately, just pull all the CSS and JavaScript libraries into dropbox itself, where they will all be hosted over https, and of course you'll have the added benefit of not having to worry about the remote CDN (content delivery networks) going down.

Anyhow, this just seems to be Chrome at the moment, and doesn't affect things like safari on iOS etc. allowing us to continue to get web apps on mobile very very fast!

Dropbox Drops Public - effect on Online Students

So it turns out the Dropbox removed the Public folder for new users a few months back.  This actually threw me and a lot of my students - for several months I have been continuing to tell my students to use their public dropbox folders, and it has only just come to light that they didn't have one unless they created it explicitly.   All good now, but it's a great example of poor visibility.  Not dropbox's fault as such.  Just something that's a problem with lots of online interactions - you can't see what the other person is looking at and you tend to assume that they see what you see.  If you've got a Public folder you assume others have.  You can say, well the students should tell you, but students are often overwhelmed and tend to assume, erroneously, that the instructor knows what they are talking about ... :-)

The Dropbox Public folder can still be created by any user who wants one and they are really really useful for quick testing of webpages, sharing images and so forth.  So useful in fact I wish I could adjust the Ctrl-Shift-4 screen capture on OSX so that any screen capture would be immediately moved to my public dropbox folder and then the public url for it be added to my clipboard, so that I could go Ctrl-Shift-4 and then Ctrl-V into an IM like Skype text chat to immediately share what I am seeing with whoever I happen to be chatting to at the time.

What's interesting to me is how long it's taken to work out that this was a problem.  The cut and paste screen shots feature that I mention above has been implemented in Linux by one of my past students, although it worked specifically with his own server and not with Dropbox.  It's a potentially great solution to the problem of poor visibility when working with others online.  Screen share via Skype or Google Groups is a great way to debug these issues, but it takes time to setup and the tendency is not to go there unless an obviously serious problem comes up.

I've noticed myself starting to prefer Skype text chat recently over email recently.  Some people have laughed about that, "yeah right, because starting up an email client is so hard", however I think there is something important there.  It's like being in the same room with someone and just being able to turn around and talk to them, versus needing to pick up the phone to call them.  To send an email I have to go to my email client (I use Thunderbird and a few Cmd-Shift clicks on OSX will take me there), get a new message window open, start typing the person's email address, and then a subject line and then get my cursor in the main window.  It's not huge, and perhaps it would be simpler in some webmail set ups, but it's all little cruft actions that get in between my thought and getting it across to someone.

For IM chat (in my case, usually Skype text chat) I just Cmd-Shift to that window and start typing.  I may need to click on a different chat room, but it's pretty damn fast.  Of course the dynamics of both email and IM change if you are actually responding to someone, but I really do find myself feeling like their is a lower barrier to the IM chat, and by arranging the groups of people in the different Skype chat rooms appropriately there's much better visibility than email, although that depends on how people are using Skype.

Some might argue that the lower barrier is a bad thing, causing myself, colleagues and students to spend all their time blathering in Skype text chat.  Also, as Dave Patterson mentions in CS169.1X, there is the danger of communication addiction, where you are constantly seeking little dopamine squirts and find yourself checking for messages all the time.

However, this issue of Dropbox dropping the public folder only finally became clear to me as a result of chatting with a student in Skype text chat.  Maybe it would ultimately have come out in an email or a forum post, and maybe it distracted me from doing something else important, but having at least a partial window into what students are going through as they try to complete assignments seems like it might be the most crucial thing for an online instructor.  Or perhaps I'm just deluding myself in order to get another dopamine fix :-)

My Mobile Design & Programming Course Available Publicly

So Hawaiʻi Pacific University (HPU) has very kindly given me permission to take my Mobile Design and Programming class public.  The class is already under way and has 9 HPU students and 6 external students.  It's not exactly a MOOC (Massive Open Online Class), more of a SOOC (Small Open Online Class).  I'm quite happy with it that way for the moment.  I'm testing out some new features such as an assignment self-tracking system (see image below) that I developed using Google App Scripts and plugged into Google Sites, and I think it's good to start small with these things.

Also, I think the key thing about classes is being able to have meaningful interactions with the other class members, which is challenging in MOOCs.  I'm hoping that I can gradually scale this class up to more students while maintaining a tight knit feel.  In order to do that I think we have to start small and gradually work our way up, rather than starting with 40,000 students and then trying to work back from there.

Some things I am happy with are having migrated my class syllabus purely to HTML (goodbye word documents), shared all my materials under Creative Commons BY licensing and being explicit about learning objectives on a week by week basis.  I'm also continuing to enjoy having all my lecture videos on YouTube, Skype chatrooms and having all my students submit assignments on their own personal blogs. I think this last is the critical thing that may allow this more personal (POOC?) type of course to scale.

The problem with MOOCs is that they hope all the students will spend time posting internally into their forums, which creates a great cache of content inside the MOOC for the MOOC authors, but I feel the value proposition is not so good for the students.  By posting on their blogs students associate their assignments and opinions more strongly with themselves; they build up a portfolio of their work that can exist, and be discovered, outside the MOOC.  The student can still control their blog visibility, and they can always go back and revise any content they are not happy with.  Rather than a certificate at the end of the course, the students have a fully browsable blog documenting their creative process - I know which I'd prefer :-)