tag:blogger.com,1999:blog-91486837078451276052024-02-18T18:33:06.039-08:00Sam's Technical BlogSomewhere for me to blather about technical stuffSam Josephhttp://www.blogger.com/profile/10788506730233381803noreply@blogger.comBlogger160125tag:blogger.com,1999:blog-9148683707845127605.post-14648118089079210032016-08-23T08:46:00.002-07:002016-08-23T08:46:54.071-07:00Pair Programming Robots and having Fun!I've had this idea for a lightweight pair programming app for a while now. One of the main inspirations is the <a href="https://itunes.apple.com/gb/app/letterpress-word-game/id526619424?mt=8">iOS letterpress app</a>. If you don't know the game it's a letter square, and you try to make the longest word you can from the letter square. Pretty straightforward stuff, except that you can play and challenge people all round the world. You take turns to make the longest word you can. What's really cool about the app is that you can have a little game with a total stranger. At least I assume you are actually playing with someone rather than against a robot. I'm not sure how drop outs are managed. When I think about it, playing against robots would be a more reliable experience, but I don't know the stats for drop outs. I've certainly dropped out of games, but perhaps I'm a sociopath and 99% of people carry on playing with each other till the end?<br />
<br />
Anyway, these days there are lots of games where you compete and even team up with strangers, e.g. Splatoon, League of Legends (LoL) and so on. I'd love to learn more about how these games match people up to try and maximise the user experience as I think we have a related problem with matching people up for pairing sessions in our "Agile Development using Ruby on Rails" MOOC. If Splatoon/LoL are the gaming equivalent of full screenshare pairing, then simpler games like letterpress would correspond to a sort of micro-pairing experience on a small toy problem.<br />
Ever since I've looked into the different styles of <a href="http://www.agileventures.org/remote-pair-programming/pair-programming-protocols">ping-pong pairing</a> I've been fascinated how protocols like "one undermanship" have a game like feel. They remind me somehow of turn based games like chess. So I keep thinking of a letterpress style micro-pair programming game where you are involved in light-weight ping-pong pairing sessions with people all round the world.<br />
<br />
Maybe there's nowhere near the number of people interested in pair programming as there are in playing word games, so maybe there would never be the critical mass to make it fun, ... unless, robots? I finally spent some time on a micro-pairing robot on a plane recently. There are a couple of challenges; one is working out the rules of this ping-pong pairing game I'm imagining, and another is getting a robot to pair program sensibly with you.<br />
<br />
An instance of a pairing game might run like this:<br />
<br />
Spec involves input/output, e.g. 2 => 4, 3 => 6, 4 => 8 (in this case a numeric doubler). The "one undermanship" protocol involves writing the absolute minimum amount of code (which we could perhaps measure in terms of characters? code-complexity?)<br />
<br />
Pair A writes:<br />
<br />
<div class="sourceCode">
<pre class="sourceCode rb"><code class="sourceCode ruby">describe <span class="st">'doubler'</span> <span class="kw">do</span>
it <span class="st">'doubles a number'</span> <span class="kw">do</span>
expect(double(<span class="dv">2</span>)).to eq <span class="dv">4</span>
<span class="kw">end</span>
<span class="kw">end</span></code></pre>
<pre class="sourceCode rb"><code class="sourceCode ruby"><span class="kw">
</span></code></pre>
</div>
Initial failing test. So Pair B works to write the absolute minimum code to make this test pass, e.g.<br />
<br />
<div class="sourceCode">
<pre class="sourceCode rb"><code class="sourceCode ruby"><span class="kw">def</span> double(number)
<span class="dv">4</span>
<span class="kw">end</span></code></pre>
<pre class="sourceCode rb"><code class="sourceCode ruby"><span class="kw">
</span></code></pre>
</div>
and then writes a new test that will fail<br />
<br />
<div class="sourceCode">
<pre class="sourceCode rb"><code class="sourceCode ruby">describe <span class="st">'doubler'</span> <span class="kw">do</span>
it <span class="st">'doubles a number'</span> <span class="kw">do</span>
expect(double(<span class="dv">2</span>)).to eq <span class="dv">4</span>
<span class="kw">end</span>
it <span class="st">'doubles a number'</span> <span class="kw">do</span>
expect(double(<span class="dv">3</span>)).to eq <span class="dv">6</span>
<span class="kw">end</span>
<span class="kw">end</span></code></pre>
<pre class="sourceCode rb"><code class="sourceCode ruby"><span class="kw">
</span></code></pre>
</div>
So pair A writes the minimum to make this pass, in this case being intentionally obtuse (they could write <code>number*2</code> which would be fewer characters, but perhaps we can give them points for passing only the existing tests and not others?):<br />
<br />
<div class="sourceCode">
<pre class="sourceCode rb"><code class="sourceCode ruby"><span class="kw">def</span> double(number)
<span class="dv">4</span> <span class="kw">if</span> number == <span class="dv">2</span>
<span class="dv">6</span>
<span class="kw">end</span></code></pre>
<pre class="sourceCode rb"><code class="sourceCode ruby"><span class="kw">
</span></code></pre>
</div>
then pair A adds another failing test:<br />
<br />
<div class="sourceCode">
<pre class="sourceCode rb"><code class="sourceCode ruby">describe <span class="st">'doubler'</span> <span class="kw">do</span>
it <span class="st">'doubles a number'</span> <span class="kw">do</span>
expect(double(<span class="dv">2</span>)).to eq <span class="dv">4</span>
<span class="kw">end</span>
it <span class="st">'doubles a number'</span> <span class="kw">do</span>
expect(double(<span class="dv">3</span>)).to eq <span class="dv">6</span>
<span class="kw">end</span>
it <span class="st">'doubles a number'</span> <span class="kw">do</span>
expect(double(<span class="dv">4</span>)).to eq <span class="dv">8</span>
<span class="kw">end</span>
<span class="kw">end</span></code></pre>
<pre class="sourceCode rb"><code class="sourceCode ruby"><span class="kw">
</span></code></pre>
</div>
And finally pair B writes the general solution:<br />
<br />
<div class="sourceCode">
<pre class="sourceCode rb"><code class="sourceCode ruby"><span class="kw">def</span> double(number)
number * <span class="dv">2</span>
<span class="kw">end</span></code></pre>
<pre class="sourceCode rb"><code class="sourceCode ruby"><span class="kw">
</span></code></pre>
</div>
Of course pair B could write:<br />
<br />
<div class="sourceCode">
<pre class="sourceCode rb"><code class="sourceCode ruby"><span class="kw">def</span> double(number)
<span class="dv">4</span> <span class="kw">if</span> number == <span class="dv">2</span>
<span class="dv">6</span> <span class="kw">if</span> number == <span class="dv">3</span>
<span class="dv">8</span>
<span class="kw">end</span></code></pre>
<pre class="sourceCode rb"><code class="sourceCode ruby"><span class="kw">
</span></code></pre>
</div>
But it would be nice if we could somehow end on the more general case, with a stronger set of tests (for edge cases etc.? could build those into the initial input/outputs). The thing about making this an enjoyable game might be a scoring system? So that you get points for one-undermanship obtuseness to an extent, but that past a certain point there's a refactoring bonus. Maybe the sensible approach is to only score a single round of hard coding where the complexity is actually less than the general solution?<br />
<br />
So there's also the issue of coming up with a range of simple coding problems that make this more interesting than the most trivial cases - I guess there's enough complexity in a few basic arithmetic problems, and we can collect more over time - there are great repositories like code wars. Anyway, with any multi-player game we have the classic bootstrap problem that if we had a great game that lots of people were playing, then there would be lots of people to play with and it would be great; but initially there are no people playing it. So in the meantime can we scaffold the gameplay with pretend people? Can we write a robot pairer than can make a test pass, and generate a new chunk of code to move the ping pong on?<br />
<br />
For a restricted set of cases I think the answer is yes. At least what I started on the plane was a chunk of code that would take and analyse a ruby exception and write the necessary code to make it pass. It's not very complex at the moment, it's basically this:<br />
<br />
<div class="sourceCode">
<pre class="sourceCode rb"><code class="sourceCode ruby"><span class="kw">def</span> fix_code(e)
<span class="kw">if</span> e.class == <span class="dt">NoMethodError</span>
<span class="ot">/undefined method \`(.*)\' for main\:Object/</span> =~ e.message
eval <span class="st">"def </span><span class="ot">#{</span><span class="dt">$1</span><span class="ot">}</span><span class="st">; 'robot method' ; end "</span>
<span class="kw">elsif</span> e.class == <span class="dt">ArgumentError</span>
<span class="ot">/wrong number of arguments \(given (.*), expected \d\)/</span> =~ e.message
num_args = <span class="dt">$1</span>.to_i <span class="co"># could use class or arg to auto-infer an approprate name?</span>
arg_string = (<span class="dv">0</span>..num_args<span class="dv">-1</span>).map {|i| <span class="st">"arg</span><span class="ot">#{</span>i<span class="ot">}</span><span class="st">"</span>}.join(<span class="st">','</span>)
<span class="ot">/\(eval\)\:1\:in \`(.*)\'/</span> =~ e.backtrace.first
method_name = <span class="dt">$1</span>
eval <span class="st">"def </span><span class="ot">#{</span>method_name<span class="ot">}</span><span class="st">(</span><span class="ot">#{</span>arg_string<span class="ot">}</span><span class="st">); 'robot method' ; end"</span>
<span class="kw">else</span>
puts <span class="st">"cannot handle error class </span><span class="ot">#{</span>e.class<span class="ot">}</span><span class="st">; </span><span class="ot">#{</span>e.message<span class="ot">}</span><span class="st">"</span>
<span class="kw">end</span>
<span class="kw">end</span></code></pre>
<pre class="sourceCode rb"><code class="sourceCode ruby"><span class="kw">
</span></code></pre>
</div>
What it does at the moment is take NoMethodErrors and ArgumentErrors and fix things up so the specified method is created with the correct number of arguments. Assuming that the game involves working through a set of input/output values on a range of basic arithmetic problems I can imagine it being fairly easy to extend to make the majority of failing tests pass. Given an input/output pair, generating an RSpec test is pretty trivial. So a little more work here and one could have a basic ping pong pairing partner. I don't fool myself that it wouldn't break fairly quickly, but I think rounds of polishing could make it work reasonable well for a range of introductory problems. Would it create a fun game that many people would want to play? Probably not ... Might it be a good learning experience for some people? ... maybe? I think the process of stack-trace/error analysis is quite interesting and a nice feature would be to have the robot be able to explain why it does what it does - they would be canned explanations, but they could highlight how the stacktrace/error has been analysed in order to work out what to do next.<br />
<br />
I guess the best initial interface would be to make it a command line game that you could play and the robot would edit the file that you are both working on perhaps? Having started it I'm kind of interested in extending it; we'll see if anyone else thinks this is anything other than mindless naval gazing :-)Sam Josephhttp://www.blogger.com/profile/10788506730233381803noreply@blogger.com0tag:blogger.com,1999:blog-9148683707845127605.post-15573716407051100522016-07-18T03:48:00.004-07:002016-07-18T03:48:51.900-07:00ReArchitecting the AutoGraderSo on Friday I followed through with my <a href="http://blog.agileventures.org/grading-the-world/">plans to get the rest of the FeatureGrader to expose errors</a> in the students’ code to students, rather than just having it respond with “your code timed out or had an error” and I think I was largely successful.<br />
<br />
At least I got those few extra code changes deployed into production and my manual tests through the edX interface showed me that my test examples would display full errors for RSpec failures, migrations failures, and Rails failures. Of course I’m blogging before I’ve reviewed how things faired over the weekend, but it feels like a step in the right direction. Even if the students can’t understand the errors themselves, they can copy and paste the output and perhaps a TA has an increased chance of helping them.<br />
<br />
I also wrapped my spike in tests like:<br />
<br />
<pre><code class="language-gherkin"> <span class="hljs-comment">@require_net_connect</span>
<span class="hljs-keyword">Scenario</span>: student submits a HW4 with migration_error
<span class="hljs-keyword">Given</span> I set up a test that requires internet connection
<span class="hljs-keyword">Given</span> an XQueue that has submission <span class="hljs-string">"hw4_migration_error.json"</span> in queue
<span class="hljs-keyword">And</span> has been setup with the config file <span class="hljs-string">"conf.yml"</span>
<span class="hljs-keyword">Then</span> I should receive a grade of <span class="hljs-string">"0"</span> for my assignment
<span class="hljs-keyword">And</span> results should include
<span class="hljs-string">"SQLException: duplicate column name: director: ALTER TABLE"</span>
</code></pre>
<br />
to check that the errors would actually be displayed even as we mutate the code. I have a selection of similar scenarios which feel like they are crying out to be DRYed out with a scenario template. Similarly, with these tests in place I wonder if I can’t improve some of the underlying grading code. Maybe we can re-throw these TestFailedError custom errors that look like they might have been intended for communicating submitted code errors back up to the grader. I found myself spending the time I could have been doing further refactoring reaching out to individual students on the forums and in GitHub to add some details about where the grader had been getting stuck for them, and encouraging them to re-submit since the grader had changed and they should now be able to see more details.<br />
<br />
I just sneaked a peak at the <a href="https://github.com/saasbook/hw-acceptance-unit-test-cycle/issues/23">GitHub comment thread</a>, and while there are some new issues that could distract me from completing this blog, at the very least I can see some students deriving value from the new level of grader feedback. So grader refactoring? I continue to feel negative about that task. The nested sandboxes of the feature grader … the fear is that refactoring could open new cans of worms and it just feels like we miss a huge chance by not having students submit their solutions via pull request.<br />
<br />
So how would a PR-based grader work? Well, reflecting on the git-immersion grader that we developed for the AV102 Managing Distributed Teams course, we can have students submit their GitHub usernames and have the grader grab details from GitHub. We can get a <a href="https://developer.github.com/v3/pulls/#list-commits-on-a-pull-request">list of comments from a PR</a> and so if we had code-climate, CI etc. set up on a repo and had students submit their solutions as pull-requests we could pull in relevant data using a combination of the repo name and their GitHub username.<br />
<br />
Making pull-requests would require students to fork rather than clone repos as they were originally encouraged to do. Switching back to that should not be a big deal. I was keen to remove forking since it didn’t really contribute to the experience of the assignment and was just an additional hoop to jump through. However if submission is by PR then we want students to understand forking and pulling; and of course that’s a valuable real world skill.<br />
<br />
This means all the solutions to the assignments exist in much larger numbers in GitHub repos, but they exist in a lot already, so not much change there. What we might have though is students submitting assignments through a process that’s worth learning, rather than an idiosyncratic one specific to edX and our custom auto graders.<br />
<br />
With a CI system like Travis or Semaphore we can run custom scripts to achieve the necessary mutation grading and so forth; although setting that up might be a little involved. The most critical step however is some mechanism for checking that the students are making git commit step by step. Particularly since the solutions will be available in even greater numbers, what we need to ensure is that students are not just copying a complete solution verbatim and submitting in a single git commit. I am less concerned about the students ability to master an individual problem completely independently, and more concerned being able to follow a git process where they write small pieces of code step by step (googling when they get stuck) and commit each to git.<br />
<br />
So for example in the Ruby-Intro assignment I imagine a step that checks that each individual method solution was submitted in a separate commit and that that commit comes from the student in question. Pairing is a concern there, but perhaps we can get the students set up so that the pairing session involves author and committer so that both are credited.<br />
<br />
But basically we’d be checking that the first <code>sum(arr)</code> method was written and submitted in one commit, and then that <code>max_2_sum(arr)</code> was solved in a separate commit, and that the student in question was either the committer or the author on the assignment. In addition we would check that the commits were suitably spaced out in time, and of a recent date. The nature of the assignment changes here from being mainly focused on “can you solve this programming problem?”, to “can you solve this code versioning issue?”. And having the entire architecture based around industry standard CI might allow us to reliably change out the problems more frequently; something that feels challenging with the current grader architecture. The current grader architecture is set up to allow the publication of new assignments, but the process of doing so is understood by few. Maybe better documentation is the key there, although I think if there is a set of well tested assignments, then the temptation for many instructors and maintainers is just to use the existing tried and tested problems and focus their attention on other logistical aspects of a course.<br />
<br />
Using existing CI systems effectively removes a large portion of the existing grader architecture, i.e. the complex sandboxing and forking of processes. This then removes a critical maintenance burden … which is provided reliably and free by the many available CI services (Travis, Semaphore, CodeShip etc.). Students now start to experience industry standard tools that will help them pass interviews and land jobs. The most serious criticism is the idea is that students won’t be trying to solve the problems themselves, but google any aspect of our assignments and find links like <a href="http://codereview.stackexchange.com/questions/91270/sums-of-some-array-elements">this</a>. The danger of the arms race to keep solutions secret is that we burn all our resources on that, while preventing students from learning by reviewing different solutions to the problem.<br />
<br />
I’m sure I’m highly biased but it feels to me that having students submitted a video of themselves pairing on the problem, along with a technical check to ensure they’ve submitted the right sorts of git commits will reap dividends in terms of students learning the process of coding. Ultimately the big win would be checking that the tests were written before the code, which could be checked by asking students to commit the failing tests, and then commit the code that makes them pass. Not ideal practice on the master branch but acceptable for pedagogical purposes perhaps … especially if we are checking for feature branches, and then even that those sets of commits are squashed onto master to ensure it always stays green …<br />
<br />
<b>Footnote:</b><br />
<br />
I also reflect that it might be more efficient to be using web hooks on the GitHub Repos in question, rather than repeatedly querying the API (which is rate limited). We’d need our centralised autograder to be storing the data about all the student PRs so that we could ensure that the student’s submission was checked in a timely fashion.Sam Josephhttp://www.blogger.com/profile/10788506730233381803noreply@blogger.com0tag:blogger.com,1999:blog-9148683707845127605.post-18614808730992136302016-07-11T09:31:00.002-07:002016-08-26T03:45:56.383-07:00Pairing - it's the Logistics Stupid!Pair programming is one of the more controversial practices associated with Agile and Extreme programming. It inspires a full spectrum of emotions from love to hate. We’ve taken a new step in the current run of the “Agile Development using Ruby on Rails” MOOC (formerly known as Engineering Software as a Service), in making pair programming a requirement of getting a certificate. There are five programming assignments in the course and we’re giving half of each assignment grade on a pair programming component. Learners submit pair programming videos and are assessed by peer review. Each learner needs to review two other pair programming videos for relevance, pairing role rotation and communication.<br />
<br />
The peer review rubric is based on the following four questions:<br />
<ol>
<li>were you programming on the relevant assignment? (1 point)</li>
<li>did you have a pair partner? (9 points)</li>
<li>did you rotate driver/navigator roles frequently? (approx evenly 4 times an hour) (40 points)</li>
<li>did you communicate effectively? i.e. regular talking, discussing the task at hand (50 points)</li>
</ol>
The rubric has evolved through several rounds of testing. The points are somewhat arbitrary, and set as they are in order to have them add up to 100 and match the 100 points that are received from the correctness of the separately submitted programming assignment code. They’re not completely arbitrary of course; having a video that shows you and a partner actually working on the correct assignment is an important gateway, and we consider good rotation and communication of higher and similar importance.<br />
<br />
There’s a conflict here of course between trying to prevent people gaming the system (submitting irrelevant or borrowed videos) and encouraging effective pair programming behaviour. The peer review process itself is not without controversy. We’ve softened the wording under rotation to indicate that roles do not need to be rotated precisely every 15 minutes, and increased the range of rotation points that can be assigned. The rotation rubric now looks like this:<br />
<blockquote>
<b>Rotation</b></blockquote>
<blockquote>
Did the participants rotate driver/navigator roles frequently? Ideally at least once every 15 minutes, i.e. this means that roughly every 15 minutes the person who is typing stops and allows their partner to type for the next 15 minutes, while switching to an advisory “navigator” role. Check particularly the time-indexes submitted.</blockquote>
<blockquote>
Don’t be too strict - the point is that the participants rotate roles approximately four times an hour - so for example a rotation at 13 mins, then after 17 mins then after 16 mins and then 14 mins is fine.</blockquote>
<ul>
<li><em>No (0 points) There was no driver navigator rotation, only a single person contributing code during the video</em></li>
<li><em>Not so much (10 points) There was only one driver/navigator rotation for hour of coding</em></li>
<li><em>A couple of times (20 points) There were a couple of driver/navigator rotations per hour of coding</em></li>
<li><em>Several times (30 points) There were 3 or even 4 rotations, but they weren’t spaced out very well over an hours coding</em></li>
<li><em>Yes (40 points) There were at least 4 rotations and all the rotations were roughly evenly spaced throughout the pairing, i.e. at least one every 15 minutes</em></li>
</ul>
We introduced more gradations than the original Yes/No in response to feedback to learners. However the other parts of the rubric are still Yes/No. We have a more graduated version of the communication rubric that we haven’t employed yet:<br />
<blockquote>
<b>Communication</b></blockquote>
<blockquote>
Did the pair continue to communicate effectively through the pairing process? Specifically, did the driver explain what they were doing as, or around, the process of them typing. Did the navigator ask questions or make suggestions or look up relevant documentation to the task at hand?</blockquote>
<ul>
<li><em>No (0 points) There was no communication or no helpful task focused communication between driver and navigator.</em></li>
<li><em>Not so much (10 points) There was almost no communication between driver and navigator, but they did communicate a little. Or the majority of communication was not helpful or task-focused.</em></li>
<li><em>Some (20 points) There were some occasional periods when the pair was communicating effectively, but there were longer periods of total silence and/or most of the communication was unrelated to the task and not helping the pair move forward.</em></li>
<li><em>A Fair Amount (30 points) There was communication but it was on and off, or communication that was unhelpful, e.g. talking off topic, getting angry etc.</em></li>
<li><em>Quite a lot (40 points) The communication was pretty good, but there was the occasional period with no communication of any kind, when perhaps it might have been helpful.</em></li>
<li><em>Yes, lots and very effectively (50 points) There was regular communication between the driver and the navigator. Although there might be occasional silences it is clear from the communication that driver and navigator are focused on solving the same task together, and they are using excellent communication skills to complete that task.</em></li>
</ul>
We’re holding off this further extension of the rubric for fears of too much complexity. There are also issues arising from how the edX peer review training component works where the learner has to match the instructed chosen grades on example videos, and so a more complex rubric leads to an even trickier peer review training process.<br />
<br />
The edX peer review system is also a little tricky for some learners since progress through the review training component is not obvious. That said there is great support for learners to give feedback on the reviews they receive, and a nice admin interface to allow us to override peer review grading where necessary. I just which I could hook it all up to a slack bot via APIs ...<br />
<br />
The peer review of pair programming videos is an experiment to see if we can effectively and reliably grade pair programming activity. Pair programming has been shown to have great benefits for learners in terms of understanding. The process of explaining technical concepts to others is hugely beneficial in terms of consolidating learning. We’ve encouraged learners in the MOOC to use the AgileVentures remote pairing system to schedule pairing events, and use a shared Gitter chat room for additional coordination.<br />
<br />
The most challenging parts of MOOC pair programming appear to be the scheduling and frustrations arising from pair programming logistics. Anecdotally there is a fair amount of time spent in Google hangouts waiting for a pair partner to appear, or conversely one’s pair partner has to leave early. Some people feel nervous about the process of pair programming with a different stranger each week.<br />
Some of this is specific to a MOOC where participation drops precipitously as the weeks go by. For the first assignments it’s easy to find a partner, but if you join the course late, or simply move on to the later assignments where fewer learners are available finding a pair partner in your timezone can be challenging.<br />
<br />
Of course your pair partner doesn’t have to be in the same timezone as you. In principle there are learners in other timezones with different schedules that happen to overlap with you. We don’t require learners to pair with others in the MOOC. They can pair with their room-mates, work colleagues, friends, whoever comes to hand. The only requirement is that a video is submitted. AgileVentures events provide a relatively convenient way to generate hangouts on air which make it fairly straightforward to generate a youtube screencast of a pairing session. Even the hangout itself can be used as a local pairing sessions recording mechanism. There are however too many points that one can get stuck at.<br />
<br />
Have we found the perfect solution? Certainly not. The difficulty with a more rigid pair scheduling system in a MOOC is not being able to rely on the participation of learners who are only involved in an ad-hoc basis. That said, my experience running a bootcamp is that an imposed pairing schedule is actually preferred by most learners, since it removes the cognitive overhead of negotiating with others about initiating pairing sessions.<br />
<br />
Perhaps for the next run of the MOOC, we could have a more structured approach where we get groups of 8 together at specific times, with the assumption that at least 2 of the 8 will show up and will be able to pair … we’ll need to analyse all the pairing data in some detail first …Sam Josephhttp://www.blogger.com/profile/10788506730233381803noreply@blogger.com0tag:blogger.com,1999:blog-9148683707845127605.post-3577813544927281162016-07-01T05:11:00.000-07:002016-07-18T04:39:18.346-07:00Delivering Features vs Testing them (and Jasmine Async)Michael was a little later to our pairing session yesterday and so I spent a little time scratching some itches about the way in which our AgileBot pings the Gitter chat for the “Agile Development using Ruby on Rails” MOOC. I’d previously been frustrated in my attempts to add hyperlinking to the Slack messages since the ‘request’ library seems to escape the <a href="https://api.slack.com/docs/message-formatting#linking_to_urls">Slack hyperlink syntax</a>, and it wasn’t immediately clear how to address that. Any changes to the AgileBot might easily blow up since it’s a legacy app with no tests. Michael and I had been trying to rectify that this week, but we were moving slowly due to lack of familiarity with the CoffeeScript AgileBot is written in; so I took the opportunity of some solo time to have a crack at reformatting the Gitter messages, which looked like this:<br />
<br />
<img alt="existing message format" height="69" src="https://www.dropbox.com/s/o22gcw1rl86i4im/Screenshot%202016-07-01%2008.37.07.png?dl=1" width="640" /><br />
<br />
No one had complained about them explicitly, but my intuition was that the long hangout link looked messy, and might be intimidating. Of course this is dangerous territory for Agile development. A common part of the Agile approach is to ensure that you are responding to customer needs, rather than incorporating necessary features on a whim. However there’s a big difference between building a system for a paying customer compared to edging a volunteer community towards critical mass. The paying customer gives you a concrete person to satisfy. In a community you’ll get suggestions, but they may not be practical and as the saying goes, they might just tell you they want a faster horse.<br />
<br />
Anyhow, the AgileVentures system has lots of little rough edges, itches I want to scratch, and scratching them makes me feel good. Remembering that Gitter supports the full markdown syntax (unlike Slack) I thought there might be a quick win and with a live chat I could easily get user feedback like so:<br />
<br />
<img alt="my suggestion in the Gitter chat" height="135" src="https://www.dropbox.com/s/xz4fa1d23w84p6k/Screenshot%202016-07-01%2008.32.30.png?dl=1" width="640" /><br />
<br />
which got some immediate feedback<br />
<br />
<img alt="feedback" height="451" src="https://www.dropbox.com/s/vml6u2sfijuoly1/Screenshot%202016-07-01%2008.52.01.png?dl=1" width="640" /><br />
<br />
and allowed us to evolve the formatting a little further<br />
<br />
<img alt="evolving the formatting" height="268" src="https://www.dropbox.com/s/cmgo4ufb9mwbvjk/Screenshot%202016-07-01%2008.53.22.png?dl=1" width="640" /><br />
<br />
The change involved a tiny reformatting to a string in the AgileBot, specifically:<br />
<br />
<pre><code class="language-coffeescript">send_gitter_message_avoid_repeats room, <span class="hljs-string">"[<span class="hljs-subst">#{req.body.title}</span> with <span class="hljs-subst">#{user.name}</span>](<span class="hljs-subst">#{req.body.link}</span>) is starting NOW!"</span>
</code></pre>
<br />
I pushed it to the staging server, tested it there, saw that it worked on our agile-bot test channel, and then pushed it live. Within vary short order we had the re-formatted messages coming into the channel:<br />
<br />
<img alt="reformatted" height="94" src="https://www.dropbox.com/s/qa5lbwmircsg70j/Screenshot%202016-07-01%2008.55.46.png?dl=1" width="640" /><br />
<br />
I hadn’t been able to remove the time element, which is an artefact not of the AgileBot, but of the main site. I got a quick <a href="https://github.com/AgileVentures/WebsiteOne/pull/1177">pull request</a> in to fix that, but that’s take a little longer to deploy.<br />
<br />
Now maybe changing these Gitter links won’t make much difference to the community in the long run. At the very least they made me feel good and motivated on a Thursday. I hope the users in the chat get a positive feeling and are maybe inspired to get more involved in the community, and my biggest hope is that as we re-run “Agile Development using Ruby on Rails” that completely new users will have slightly less friction as they consider joining a pairing session.<br />
<br />
I’m particularly glad that I spent the time solo-ing on the above, because the rest of the afternoon pairing with Michael was somewhat frustrating in that we were repeatedly stuck on getting a basic test of the 3rd party HTTP connection for the AgileBot. We did make progress over the course of the session, but nothing that any end user would see soon. The AgileBot hits 3rd party services Gitter and Slack to get its job done. If we are going to do proper integration tests these services need to be stubbed. We trying the node equivalent of <a href="https://github.com/vcr/vcr">VCR</a>, <a href="https://github.com/linkedin/sepia">Sepia</a> by the folks at LinkedIn, which will record and save 3rd party HTTP interactions and allow them to be played back, effectively sandboxing an app. We got sepia working, however in playback mode it highlighted cache misses (unexpected network connections) by creating a file rather than throwing an error that could be caught by JasmineNode.<br />
<br />
We set that aside and tried <a href="https://github.com/node-nock/nock">Nock</a>, one of many node world equivalents of <a href="https://github.com/bblimke/webmock">WebMock</a> that allows precision stubbing of network connections. Personally I prefer approaches like VCR and Sepia that allow blanket recording of interactions, in contrast to WebMock and Nock which require you to write out individual network stubs by hand. We had nock working, but ran up against node async issues. The JasmineNode tests were not waiting for the HTTP connections to complete, and we were tying ourselves in knots trying to get the JasmineNode async to work in CoffeeScript.<br />
<br />
We untied ourselves by dropping back to ground truth, by first getting the example Jasmine Async test working in pure JavaScript:<br />
<br />
<pre><code class="language-javascript">describe(<span class="hljs-string">"Asynchronous specs"</span>, <span class="hljs-function"><span class="hljs-keyword">function</span>(<span class="hljs-params"></span>) </span>{
<span class="hljs-keyword">var</span> value;
beforeEach(<span class="hljs-function"><span class="hljs-keyword">function</span>(<span class="hljs-params">done</span>) </span>{
setTimeout(<span class="hljs-function"><span class="hljs-keyword">function</span>(<span class="hljs-params"></span>) </span>{
value = <span class="hljs-number">0</span>;
done();
}, <span class="hljs-number">1</span>);
});
it(<span class="hljs-string">"should support async execution of test preparation and expectations"</span>, <span class="hljs-function"><span class="hljs-keyword">function</span>(<span class="hljs-params">done</span>) </span>{
value++;
expect(value).toBeGreaterThan(<span class="hljs-number">0</span>);
done();
});
});
</code></pre>
That working we converted to CoffeeScript and confirmed that that worked<br />
<br />
<pre><code class="language-coffee">describe <span class="hljs-string">'Asynchronous specs'</span>, <span class="hljs-function">-></span>
value = <span class="hljs-literal">undefined</span>
beforeEach (done) ->
setTimeout (<span class="hljs-function">-></span>
value = <span class="hljs-number">0</span>
done()
), <span class="hljs-number">1</span>
it <span class="hljs-string">'should support async execution of test preparation and expectations'</span>, <span class="hljs-function"><span class="hljs-params">(done)</span> -></span>
value++
expect(value).toBeGreaterThan <span class="hljs-number">0</span>
done()
</code></pre>
then carefully inserted the elements from our AgileBot HTTP testing setup:<br />
<br />
<pre><code class="language-coffee">nock = <span class="hljs-built_in">require</span>(<span class="hljs-string">'nock'</span>);
slack = nock(<span class="hljs-string">'https://slack.com'</span>)
.post(<span class="hljs-string">'/api/chat.postMessage'</span>)
.reply(<span class="hljs-number">200</span>, {
<span class="hljs-attribute">ok</span>: <span class="hljs-literal">false</span>,
<span class="hljs-attribute">error</span>: <span class="hljs-string">'not_authed'</span>
});
avHangoutsNotifications = <span class="hljs-built_in">require</span>(<span class="hljs-string">'../scripts/av-hangouts-notifications.coffee'</span>)
describe <span class="hljs-string">'AV Hangout Notifications'</span>, <span class="hljs-function">-></span>
beforeEach ->
routes_functions = {}
avHangoutsNotifications({<span class="hljs-attribute">router</span>: { <span class="hljs-attribute">post</span>: <span class="hljs-function"><span class="hljs-params">(s,f)</span> -></span> routes_functions[s] = f } })
<span class="hljs-property">@routes_functions</span> = routes_functions
describe <span class="hljs-string">'hangouts-video-notify'</span>, <span class="hljs-function">-></span>
beforeEach (done) ->
res = {}
res.writeHead = <span class="hljs-function">-></span> {}
res.end = <span class="hljs-function">-></span> {}
req = { <span class="hljs-attribute">body</span>: { <span class="hljs-attribute">host_name</span>: <span class="hljs-string">'jon'</span>, <span class="hljs-attribute">host_avatar</span>: <span class="hljs-string">'jon.jpg'</span>, <span class="hljs-attribute">type</span>: <span class="hljs-string">'Scrum'</span> } }
req.post = <span class="hljs-function">-></span> {}
<span class="hljs-property">@routes_functions</span>[<span class="hljs-string">'/hubot/hangouts-video-notify'</span>](req,res)
setTimeout (<span class="hljs-function">-></span>
done()
), <span class="hljs-number">3000</span>
it <span class="hljs-string">'should support async execution of test preparation and expectations'</span>, <span class="hljs-function"><span class="hljs-params">(done)</span> -></span>
expect(slack.isDone()).toBe(<span class="hljs-literal">true</span>)
done()
</code></pre>
and this would just crash the whole thing. So at least we had identified that the problem was with our code, not how we happened to be implementing Async testing in CoffeeScript in JasmineNode. It was precisely this line <code>@routes_functions['/hubot/hangouts-video-notify'](req,res)</code> that was causing the crash. The one that started the network connection. Having not yet set up for an interactive debugger it was console.log statements all the way down to discover that it was the error reporting <code>roller</code> component under our hood that was actually breaking everything. Clearly that needed to be disabled for testing purposes. That was achieved like so:<br />
<br />
<pre><code class="language-coffee">rollbar.init(process.env.ROLLBAR_ACCESS_TOKEN, {<span class="hljs-attribute">enabled</span>: <span class="hljs-literal">false</span>})
</code></pre>
<pre><code class="language-coffee">
</code></pre>
and suddenly all the tests went green. It was a frustrating process, but it highlights the problem solving approach of breaking out the different elements of the system you are testing in order to isolate where the problem actually exists. We’ve achieved a much better understanding of the legacy app as a result of all this. I’m just glad for motivations sake that I kicked out a minor improvement direct to the users before we spent the afternoon wrestling with testing frameworks :-)Sam Josephhttp://www.blogger.com/profile/10788506730233381803noreply@blogger.com0tag:blogger.com,1999:blog-9148683707845127605.post-8320750117820520272016-06-28T03:13:00.000-07:002016-06-28T03:14:35.780-07:00Critical Mass of a CommunityThe holy grail of the Agile Ventures community, and perhaps any community, is to achieve "Critical Mass". That's the point at which the community becomes self-sustaining and activity proceeds without relying on any one particular individual to keep it going. "Critical Mass" is a term from physics which describes the threshold weight of nuclear material required to create a nuclear explosion.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://dl.dropboxusercontent.com/u/13196858/critical_mass.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="213" src="https://dl.dropboxusercontent.com/u/13196858/critical_mass.jpg" width="320" /></a></div>
<br />
<br />
In nuclear material it's the movement of particles called "neutrons" that cause individual atoms (in particular the atomic nuclei) to split apart, or undergo what's called Nuclear Fission. What makes a nuclear explosion possible is that this process of fission releases additional neutrons, which can go on and cause other atoms to split apart. If you have a large enough amount of the right material it's almost inevitable that each neutron generated will collide with another atom as it travels through the material, which generates more neutrons which collide with other atoms and so on. This is called a chain reaction. Have too little material and the neutrons will be leave the material without having hit other atoms, and the chain reaction dies out.<br />
<br />
Let's explore the analogy with a community, in particular a pair programming community. Each pairing session could be considered an atom. Assuming the you have one pairing session take place (and it goes reasonably well), you'll end up with two people who are interested in pairing again. They'll then be searching for other pairing sessions, but if there are none available, or none that they happen to be interested in (wrong programming language or platform) then it's likely these two will drift off and perhaps not try to pair in the same community again. However if these two do find other pairing sessions, you can see how the single successful pairing event can lead to two more. Assuming those sessions go well, you have four people now looking to pair and so on.<br />
<br />
Under the right conditions you can get a chain reaction. It requires a critical mass of people taking part in pairing sessions. Ideally whenever anyone wants to find a pair, there is always someone there ready to go. Of course all this depends on people being able to find and join pairing sessions and also for them to go well.<br />
<br />
Too few people and there's just not that many opportunities for pairing; but lots of people is not enough. Imagine that lots of people are trying to pair but that problems with the interface mean that people trying to join a pairing session end up in the wrong location. No pair partner, no pairing. Michael and I uncovered one problem with the AgileVentures interface last week. Hangouts that had people in them were being reported as "not live" after 4 minutes. This meant that on a fair number of occasions people attempting to join a hangout for pairing or for a meeting would find themselves on their own in a separate hangout.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://www.dropbox.com/s/utp0der29vwvb1j/upcoming_events.png?dl=1" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="154" src="https://www.dropbox.com/s/utp0der29vwvb1j/upcoming_events.png?dl=1" width="320" /></a></div>
<br />
<br />
We've just rolled out a fix for this and hopefully this will be another step towards achieving critical mass in the community. It's unlikely to be the only step required as having a good pairing experience is more complex than nuclear fission. We also want to adjust our user experience to maximise the chances of a good pairing experience for everyone. It's not clear the best way to do that but clearly getting two people into the same hangout at the same time is an important pre-requisite. Things that we're exploring include adding automated "pair rotation" timers to the hangout itself; having users rate their pairing experience; reflecting pairing activity through user profiles and so on.<br />
<br />
We need to carefully monitor the changes and fixes we just made to see how the proportion of pairing event participation changes, and continue our Agile iterative process of making small changes and reflecting on their effect. Making it more obvious which events are live might lead to more disruption in pairing events, or it might make observing ongoing pairing events easier, and that might make people more or less inclined to initiate their own pairing events. It's not simple, but with careful measurement hopefully we can find that sequence of changes to the user experience that will lead to critical mass!Sam Josephhttp://www.blogger.com/profile/10788506730233381803noreply@blogger.com0tag:blogger.com,1999:blog-9148683707845127605.post-20347635268293912062016-06-24T02:05:00.004-07:002016-06-24T02:08:57.251-07:00Analyzing Live Pair Programming DataThe recent focus for the AgileVentures (WebSiteOne) development team is trying to make the process of joining an online pair programming as smooth as possible. The motivation is two-fold; One, we want to have our users to have a good experience of pairing, and all the learning benefits it brings; Two, we want to have large numbers of users pairing so that we can get data from their activity to analyse. The latter motivation sort of feeds the first one really, since the point of analysing the data is to discover how we can serve the users better, but anyhow ... :-)<br />
<br />
Several months back we created an <a href="https://github.com/AgileVentures/WebsiteOne/issues/874">epic</a> on our waffle board that charted the flow from first encountering our site to taking part in a pairing session. We identified the following key components:<br />
<ol>
<li>Signing Up</li>
<li>Browsing existing pairing events</li>
<li>Creating pairing events</li>
<li>Taking the pairing event live</li>
<li>Event notifications</li>
<li>Event details (or show) page</li>
<li>Editing existing events</li>
</ol>
The sequence is only approximate as signing up/in is only required if you want to create an event, and not required for you to browse and join events. The important thing was that there were various minor bugs blocking each of the components. We set about trying to smooth the user experience for each of the components, including sorting out <a href="http://blog.agileventures.org/agile-approach-to-login-signup-problems/">GitHub and G+ signup/signin issues</a>, providing filtering of events by project, setting appropriate defaults for event creation and ironing out bugs from event edit and update, not to mention delivering support for displaying times in the users timezone, and automatically setting the correct timezone based on the user's browser settings.<br />
<br />
There are still other points that could be smoothed out, but we've done a good portion of the epic. The question that partly troubles me now is how to "put it to bed". A new epic that contains only the remaining issues is probably the way to go, but finally we've got to the point to start analysing some data, since we've got the notifications for the edX MOOC pairing activity flowing to the MOOC Gitter chat fairly reliably and we've just broken through on removing key confusions about joining an event, and working out some problems about the event displaying whether it is live.<br />
<br />
This last element is worth looking at in a little more detail as it strongly affects the type of data we are gathering. Creating (and tracking) Google Hangouts for pairing from the AgileVentures site involves creating a Google Hangout that has a particular plugin, called <a href="https://github.com/AgileVentures/HangoutConnection">HangoutConnection</a>, that knows the server side event it is associated with. This was originally designed by <a href="http://www.agileventures.org/users/yaro-apletov">Yaro Apletov</a> and is written in CoffeeScript. It gets loaded when the hangout starts and attempts a connection back to the main AgileVentures site. Given successful contact an EventInstance object is created in association with the event. This EventInstance includes information about the hangout such as the URL, so that other people browsing the site can also join the hangout without being specifically invited. The HangoutConnection continues to ping the site every two minutes assuming the hangout is live, the plugin hasn't crashed and so on.<br />
<br />
What Michael and I identified on Wednesday was that only the first of these pings actually maintained the live status, making it look like all our pairing hangouts were going offline after about 4 minutes. This had been evidenced by the "live now" display disappearing from events somewhat sooner than appropriate. This might seem obvious, but the focus has been on fixing many other pressing issues and usability concerns from the rest of the epic. Now that they are largely completed this particular problem has become much clearer (also it was obscured for the more regular scrums which use a different mechanism for indicating live status). One might ask why our acceptance tests weren't catching this issue. The problem here was that the acceptance tests were not simulating the hit of the HangoutConnection to our site. They were manipulating the database directly, thus as is often the case, the place where the bug occurs is just in that bit that wasn't covered by a test. Adjusting the tests to expose the problem made the <a href="https://github.com/AgileVentures/WebsiteOne/pull/1143">fix was relatively straightforward</a>.<br />
<br />
This is an important usability fix that will hopefully create better awareness that hangouts are live (with people present in them), and increase the chances of people finding each other for pairing. There's a lot more work to do however, because at the moment the data about hangout participants that is sent back from HangoutConnection gets overwritten at each ping. The Hangout data being sent back from HangoutConnection looks like this:<br />
<blockquote class="tr_bq">
<br />
{<br />
"0" => {<br />
"id" => "hangout2750757B_ephemeral.id.google.com^a85dcb4670",<br />
"hasMicrophone" => "true",<br />
"hasCamera" => "true",<br />
"hasAppEnabled" => "true",<br />
"isBroadcaster" => "true",<br />
"isInBroadcast" => "true",<br />
"displayIndex" => "0",<br />
"person" => {<br />
"id" => "123456",<br />
"displayName" => "Alejandro Babio",<br />
"image" => {<br />
"url" => "https://lh4.googleusercontent.com/-p4ahDFi9my0/AAAAAAAAAAI/AAAAAAAAAAA/n-WK7pTcJa0/s96-c/photo.jpg"<br />
},<br />
"na" => "false"<br />
},<br />
"locale" => "en",<br />
"na" => "false"<br />
}<br />
}</blockquote>
<br />
Basically the current EventInstance will only store a snapshot of who was present in the hangout the last time the HangoutConnection pinged back; and data from pings after the first two minute update has been being discarded. We're about to fix that, but here's the kind of data we can now see about participation in hangouts:<br />
<br />
<blockquote class="tr_bq">
#participants #hangouts<br />
Monday:<br />
1 *<br />
Tuesday:<br />
1 *<br />
Wednesday:<br />
1 ****<br />
2 *<br />
3 **<br />
Thursday:<br />
1 ****<br />
Friday:<br />
1 *<br />
2 *<br />
3 *<br />
Saturday:<br />
1 *<br />
Sunday:<br />
1 ***************<br />
2 *<br />
3 *<br />
Monday:<br />
1 ******************************<br />
2 ****<br />
3 <br />
4 **</blockquote>
<br />
The above is just a snapshot that corresponds to the MOOC getting started; we're working on a better visualisation for the larger data set. We can see a clear spike in the number of hangouts being started, and a gradually increase in the number of hangouts with more than one participant, remembering that the participant data is purely based on who was present at two minutes into the hangout.<br />
<br />
If the above data was reliable we might be saying, wow we have a lot of people starting hangouts and not getting a pair partner. That might be the case, but it would be foolish to intervene on that basis using inaccurate data. Following the MOOC chat room I noticed some students at the beginning of the course mentioning finding hangouts empty, but the mood music seems to have moved towards people saying they are finding partners; and this is against the backdrop of all the usability fixes we've pushed out.<br />
<br />
To grab more accurate participation data we would need to do one or more of the following:<br />
<ol>
<li>adjust the EventInstance data model so that it had many participants, and store every participant that gets sent back from the HangoutConnection</li>
<li>store the full data sent back from every HangoutConnection ping</li>
<li>switch the HangoutConnection to ping on participant joining and leaving hangouts rather than just every two minutes</li>
<li>ruthlessly investigate crashes of the HangoutConnection</li>
</ol>
With reliable data about participation in pairing hangouts we should be able to assess some objective impact of our usability fixes as they roll out. We might find that there are still lots of hangouts with only one participant, in which case we'll need to investigate why, and possibly improve awareness of live status and further smooth the joining process. We might find that actually the majority of hangouts have multiple participants, and then we could switch focus to a more detailed analysis of how long participants spend in hangouts, getting feedback from pair session participants about their experience, and moving to display pairing activities on user profiles to reward them for diligent pairing activities and encourage repeat pairing activities.<br />
<br />
Personally I find this all intensely fascinating to the exclusion of almost everything else. There's a real chance here to use the data to help adjust the usability of the system to deliver more value and more positive learning experiences.<br />
<br />
<br />Sam Josephhttp://www.blogger.com/profile/10788506730233381803noreply@blogger.com0tag:blogger.com,1999:blog-9148683707845127605.post-79780258754454012142016-06-20T00:54:00.001-07:002016-06-29T02:46:52.611-07:00Moving Beyond Toy ProblemsWhat is life for? To follow your passion. What is my passion? I find myself frustrated with the closed source, closed development nature of many professional projects; and on the flip side equally frustrated with the trivial nature of academic "toy" problems designed for "learning".<br />
<br />
I love the "in principle" openness of the academic sphere and the "reality bites" of real professional projects. I say "in principle" about academic openness, because while the results of many experiments are made freely available, sharing the source code and data (let alone allowing openness as to the process) is often an afterthought if it is there at all. The MOOC revolution has exposed the contents of many university courses which is a fantastic step forward, but the contents themselves are often removed from the reality of professional projects, being "toys" created for the purpose of learning.<br />
<br />
Toy problems for learning makes sense if we assume that learners will be intimidated or overwhelmed by the complexity of a real project. Some learners might be ready to dive in, but others may prefer to take it slow and step by step. That's great - I just don't personally want to be spending my time devising toy problems, or at least not the majority of my time. Also it seems to me that the real learning is the repeated compromises that one has to make in order to get a professional project out the door; balancing the desire for clarity, maintainability, readability and craftsmanship against getting features delivered and actually having an impact that someone cares about.<br />
<br />
Professional projects are typically closed source, closed development; although there are more and more open source projects in the professional sphere; the basic idea seems to be: we are doing something special and valuable, and we don't want you to see our secret sauce, or the mistakes we are making along the way. Thus it might be considered anti-competitive for a company to reveal too much about the process it uses to develop its software products. That said, companies like ThoughtBot publish their <a href="https://playbook.thoughtbot.com/">playbook</a>, giving us an insight into their process and perhaps increasing our trust that their process is a good one. Even so we don't get to see the "actual" process, and so that's not ideal for others trying to learn, but then most companies are not trying to support the learning process for those outside.<br />
<br />
Personally I want to have a global discussion that everyone can take part in, if they want to. I want an informed debate about the process of developing software where we have <a href="http://blog.agileventures.org/agile-approach-to-login-signup-problems/">real examples from projects</a> - real processes - where we can all look at what actually happened rather than relying on people's subjective summaries.<br />
<br />
Maybe this is just impossible, and an attempt at the pure "open development" process of <a href="http://www.agileventures.org/">AgileVentures</a> is destined to fail because by exposing exactly how we do everything we can't build up value to sustain our project? That's what professional companies do right? They have a hidden process, focus attention on the positive results of that process and then increase the perception that they have something worth paying for. To the extent that they are successful they are building up reputation that will sustain them with paying customers, because those customers are inclined to believe the chance is good they'll get value for money.<br />
<br />
If the customer had total transparent access to every part of what goes on, they could just replicate it themselves right? :-) Or a competitor could provide the same service for less? However there's a strength in openness - it shows that you believe in yourself and you demonstrate that you've followed the hard path through the school of knocks and maybe you are the right people to be able to adapt fast to the next change, even if others could copy aspects of what you do.<br />
<br />
Everyone should have the opportunity to learn and boost their skills by taking part in such an open process. The shy might not want to participate directly, but they can learn by observing a real process that they won't have till they can actually start in a job. It's the old catch-22 "no job, no experience; no experience, no job".<br />
<br />
This is what I stand for, what <a href="http://www.agileventures.org/">AgileVentures</a> stands for. An open process beyond open source, we call it "Open Development". Meetings with clients, planning meetings, coding sessions, everything is open and available to all. Where customers have confidential data, that is hidden, but otherwise we are as open as possible. Of course that generates too much material for anyone to absorb, and we need help curating it, but the most amazing learning happens when new folk engage and take part in the process - thinking about the code in their pull requests in relation to the needs of a non-technical client being articulated by a project manager who cares about their learning, but also about the success of the project. Come get involved, be brave, be open and get that experience that you can't get without a job, or that you can't get in your current job.Sam Josephhttp://www.blogger.com/profile/10788506730233381803noreply@blogger.com0tag:blogger.com,1999:blog-9148683707845127605.post-75918397389809285312014-06-13T03:14:00.002-07:002016-06-20T00:25:58.604-07:00Thoughts about pair roulette, pairing in MOOCOne pedagogical argument is that pairing should be restricted to just pairs. By having only two people in the pairing session we increase the likelihood that both are doing active work. Others may benefit from observing, but they are also reducing the pool of other possible pair partners, and in the case where students are pairing on homeworks that are being assessed, there might be an argument to suggest that observers are freeloading.<br />
<br />
However, taking an active part in a pair is very intimidating for many and so it's a big advantage in some ways if random pairing sessions are visible for observation and/or some degree of interaction with a wider group of learners.<br />
<br />
There is a tension between the desire for each individual learner to operate in the manner that feels most comfortable at any given time (working solo, working actively in a pair, observing a pair etc.) and the desire to assess learner's abilities. If we were only concerned with promoting learning, then we would not necessarily impose restrictions on observers, although there is the further case of where learners would like to pair, but would like to restrict who is observing for privacy reasons. Privacy reasons are unclear to the current author, but seem to revolve around the fear of aspects of one's personality being displayed for others to judge? Online remote pair programming does not require that people display video of themselves, or even necessarily any audio, but still one's typing is exposed in real time, and one's decisions about what to type next, what code to create, are being exposed, in the much the same way that they might be in an oral exam, and many people are intimidated by interviews and oral exams. Superficially this is related to a fear that one will be judged as not having made the grade, but the author would love to hear insights from others on this.<br />
<br />
Ideally learning situations such as remote pairing should not revolve around judgement of each others abilities, and provide a supportative environment for learning, but naturally all learners will be making judgements of each other (and themselves), such as relating to their partners language ability, coding ability, and so forth. Some learners are in a hurry and may feel that they don't have time to be pairing with someone they consider as being less skilled in areas they want to improve in.<br />
<br />
The author would suggest that the idea of placing learners on a linear spectrum of ability does not make much sense. Everyone's understanding and skills are a complex multidimensional entity. For example one individual may be very confident in Ruby String manipulation, but much less confident as regards OOP and another learner's understanding/confident may be exactly the opposite. As regards coding one's number of years programming is often considered a guide towards "ability level" but still there is no clear linear relationship, and learners might well be advised to be patient and discover what they can learn from every possible pair partner.<br />
<br />
Having all pairing sessions recorded is arguably a good move since it allows analysis of the pairing, ensure that any disputes about behaviour can be resolved with relation to a video of what actually happened. Knowledge that one is being recorded (if not directly observed) serves as an incentive to behave more congenially. The downside of recording is that it will increase the nervousness of some, and might prevent them from participating in the pairing session as actively as they would like.<br />
<br />
Assuming that one can host pairing sessions and drop 2 or more people into them, and record them; the question arises about the best mechanism of pairing people up. One of the key issues seems to be people's shifting schedules and being ready to pair. If you add your name to a list of people wanting to pair, will you be available at the time someone else is ready?<br />
<br />
If you have some mechanism for checking that someone is actually looking at a given page you could indeed have a list of people ready to go, although people may still be AFK (away from keyboard). It's an interesting question about whether people should be asked to register their interests for a pairing session, e.g. Homework 2, Project X, or open to suggestions. Should you have a single list of everyone ready for impromptu pairing, or should their be some set of lists divided up at some granularity - it seems like that depends on the numbers involved. In the first instance perhaps one should just have a single list, until one demonstrates the need for more ...<br />
<br />
There is also the question about whether one should have a list of ongoing sessions that others can browse and then join. In the open source project case it seems the answer should be a resounding yes. In the student case it seems we are caught between the desire to showcase the system running in progress (and allowing others to learn by observing), and the desire to afford students with additional privacy, and reduce perceptions of freeloading.<br />
<br />
If one is offering a service for free then one can argue that participants have a degree of obligation to share what they are doing, i.e. contributing back, although of course the flip side is that if participants feel uncomfortable then they will just not participate. It seems reasonable to suggest that a premium version of the system might allow privacy; since participants are contributing to the system by paying for it.<br />
<br />
Finally there is the question of how to effectively scaffold pairing sessions with novice pair partners. One can imagine step by step pairing walkthroughs, but it is currently unclear what the best mechanism would be to insert these into a pairing session? Perhaps a cloud9/nitrous session where the computer would prompt the pair partners to switch roles and have certain sorts of discussions at certain times. A human coach can scaffold that with some degree of skill - how much of it can we automate?Sam Josephhttp://www.blogger.com/profile/10788506730233381803noreply@blogger.com5tag:blogger.com,1999:blog-9148683707845127605.post-32751277286468663832013-07-29T17:33:00.000-07:002013-07-29T17:42:05.994-07:00Sam Joseph on MOOCs for Hawaii Business MagazineI was recently interviewed by Hawaii Business Magazine's Pavel Stankov on the subject of Massive Open Online Classes or MOOCs<br />
<br />
<b>Pavel: So first off, tell me about the public class that you taught
online at HPU? What class was it? How long was it taught?</b>
<br />
<b><br /></b>
<i>Sam: The class was the combined CSCI 4702 Mobile programming and MULT 4702 Mobile Design classes, and was taught over the usual
14 weeks of the HPU Spring semester this year. It's a class
that I've been teaching in one form or another at HPU and UHM
for about 8 years, and focuses on the design and programming of
applications for mobile devices such as tablets and smartphones.</i><br />
<br />
<b>Pavel: Who pitched the idea for that and how successful was the
course? Did anybody drop out?</b>
<br />
<br />
<i>Sam: I designed the course myself and I pitched the idea that HPU
should trial it as a public beta. I would say the course was
moderately successful :-) I continue to get the positive feedback that I have gotten from
students on this course as I have over the years. All of the official HPU students
enrolled in the course completed it, and the public format enabled some who had
previously failed the course to</i><br />
<i>
retake and successfully complete the course. The majority of
the non HPU students taking the course did not complete it, but
I still got positive feedback from them. I don't know that "dropping out" as a concept is particularly useful for
students who are not studying for credit. If for-credit students "drop out" it is clearly a
negative event where a student has paid for support in their learning process, and for
whatever reason, feels that they are not getting what they
expect from a course.</i><br />
<br />
<i>
When casual "MOOC" students are taking a course it is much more
like they are receiving an encyclopedia. They are gaining access to a set of materials
that they can take and pick from as they please. Since in the HPU public course trial we
were not certifying their abilities I don't believe that their
"drop out" rate of non-HPU students indicates the relative success or failure of the course.</i><br />
<br />
<b>
Pavel: Would it be considered a MOOC, if it's not delivered
through Coursera, edX, or Udacity? Was there a third party at
all, or was it offered directly from HPU, just open to the
public?</b>
<br />
<b><br /></b>
<i>Sam: The first MOOCs were offered before Coursera, EdX or Udacity
existed so I don't think that</i><br />
<i>
who provides a course says anything about whether it should be
considered a MOOC or not.</i><br />
<i>
Whether we consider something a MOOC or not depends on four
things, specifically it being</i><br />
<i>
a course, it being available online, it being available to the
public, and it being taken </i><br />
<i>
by a large number of students. Our online public course trial
had around 20 students combined so I think it qualifies more as
a SOOC (Small Open Online Course) than anything else :-) I used
the free and open Google Sites framework to host and deliver the
course along with other free open source tools and Google App
Scripts that I programmed myself.</i><br />
<br />
<b>
Pavel: What class are you currently teach through edX? What is the
turnout? What are the expectations and what do you hope to
achieve through offering it? Have you offered a course of such a
scale before? How do you feel about it? </b>
<br />
<b><br /></b>
<i>Sam: In collaboration with UC Berkeley's Professors Dave Patterson
and Armando Fox I am facilitating CS169X Software as a Service
through EdX. The current instance of the class</i><br />
<i>
has over 13000 students enrolled and close to 250 teaching
assistants. The expectation</i><br />
<i>
is that we can spread as widely as possible the concepts of
software engineering craftsmanship. My personal hopes are that
by being closely involved in this course that my own HPU
Software Engineering course will become even more valuable to
the students taking it, in terms of the quality of the
curricular materials they have access to, and the range of other
learners they can interact with. </i><br />
<br />
<i>Sam: One of the key values of delivering a for-credit class publicly
with a mix of for-credit and casual students is that the
students get to mix with a much wider range of learners.
Students can take part in collaborative learning with people
from all over the world, who in many cases bring fantastic
industry experience with them to the class. My involvement in
this summer's EdX course is the first time that I have taken a
major role in a class of this scale and I am extremely excited
about it. I see the combination of MOOC delivery systems such
as EdX with personal scaffolded collaborative learning
experiences such as pair programming and group projects
revolutionizing the nature of the educational experience.</i><br />
<br />
<b>
Pavel: What are the challenges for a MOOC instructor? What is the
hardest part? The easiest part?</b>
<br />
<b><br /></b>
<i>Sam: The challenges are having a continous flow of information with
questions from students coming in constantly 24/7. All
materials have to be of exceptionally high quality. The </i><i>hardest part is often just switching off for a moment to refresh
yourself. Delegation is the combined hardest and easiest part.
Given the 250 or so teaching assistants I have to control myself
not to dive too deep on problems from individual students the
moment they come up, leaving the teaching assistants to triage
the challenges the students are facing. The MOOC instructor
must listen carefully to their teaching assistants balancing
when to jump in with their expertise so as to benefit the
maximum number of students possible.</i><br />
<br />
<b>
Pavel: If you were to start over, and change something in the way
you approach your MOOCs, would you do so?</b>
<br />
<b><br /></b>
<i>Sam: I am making an ongoing effort to change MOOCs so that they move
away from the one size </i><br />
<i>
fits all mode of learning, and focus on the individual. I think
real time interaction</i><br />
<i>
with fellow MOOC students and open-ended projects of consequence
to the individual </i><br />
<i>
students is the key. Certain Stanford Coursera classes have
trail-blazed in this regard such as Scott Klemmer's HCI course,
however I think we can go a lot lot further.</i><br />
<br />
<b>
Pavel: How do you address the issues of academic honesty?</b>
<br />
<b><br /></b>
<i>Sam: Academic honesty is an issue in a class that tries to offer
credit for a largely one size fits all method of assessment. My
personal approach to academic honesty is to award credit for
unique individual contributions. To the extent possible within
the contexts of </i><i>HPU, Berkeley and EdX I make all my classes dependent on a
students ability to offer unique individual project work. In a
class where projects are developed incrementally it quickly
becomes clear if an individual is trying to attribute the work
of others to themselves, even in the online context. This is
of course more complicated at large scales, but I believe we
have the seeds in place to make academic honesty effectively a
non-issue, and you are likely to see some very exciting
developments in this regard over the next 24 months or so.</i><br />
<br />
<b>
Pavel: What is your take on the peer-grading approaches offered by
some MOOC providers? Do you suppose there might be a better
technology to control for plagiarizing? </b>
<br />
<b><br /></b>
<i>Sam: I think peer-grading is an interesting approach. I've used it
in some of my classes, and I've taken MOOC classes in which it
has been used. I don't think it is yet quite in a form that
delivers an ideal learning experience but it's an excellent
start. Since I teach programming and design, plagiarising is
not quite the issue that is in other classes such as English and
History, however it is still a concern. I have also been taking
a very interesting MOOC on cheating in online classes, and I
think it's a mistake to focus on plagiarism and how to control
it. I believe the focus should be on igniting the imagination
of individual students. The key is for the instructor or
teaching assistant to get to know the individual students
personally, and ask them what it is they are really excited
about doing. The focus should be on providing a framework, scaffolding if
you will, that enables the student to do something that they are
excited about, giving them access to the tools that allow them to achieve their dreams. I
would argue that plagiarism and academic dishonesty comes
largely from students not being interested in performing the
academic exercises they are being set. The solution is not to
ask how to control plagiarism through technology, but what is it
that interests an individual student? Unlock the interest of
the individual student and they will have no incentive to be
academically dishonest; they will be truly motivated to create
something of quality, and developing the necessary skills to
support that.</i><br />
<br />
<b>
Pavel: Do you think some classes are better suited to be taught
through a MOOC than others? Which ones?</b>
<br />
<b><br /></b>
<i>Sam: Given the form of the current MOOCs from providers such as
Coursera, EdX and Udacity, I would say that computer science
courses are particularly well suited to being taught through a
MOOC. One might argue that introductory courses are also well
suited to MOOCs, however I am not sure I agree. I think the
being suited to a MOOC depends on who's perspective we are
talking about, e.g. educator, student, institution etc. However
with the new technologies that are rolling out in terms of
remote collaboration software such</i> <i>as Google hangouts, Multiway Skype screenshare etc. the majority
of classes can be taught just as well in MOOC format, if not
better than at a physical institution. Although a clear
exception would be those courses that are attempting to train or
instruct in the use of expensive specialist equipment that is
not available to the individual at their home.</i><br />
<br />
<b>
Pavel: Under what circumstances should colleges and universities
award academic credit for third-party MOOC providers?</b>
<br />
<b><br /></b>
<i>Sam: At the moment I think it's not in colleges and universities
interest to award academic credit for third-party MOOC providers
unless assessment has been verified through proctored
examinations.</i><br />
<br />
<b>
Pavel: How do you see the development of MOOCs in the observable
future? </b>
<br />
<b><br /></b>
<i>Sam: Difficult to say although my personal hope and plan is that
MOOCs will become a lot more accessible and personal with MOOCs
supporting students to talk in real time with their peers and
instructors.</i><br />
<br />
<b>
Pavel: Is there anything in particular about Hawaii that makes our
location different when it comes to online higher education?</b>
<br />
<b><br /></b>
<i>Sam: Hawaii of course has a large military presence; and military students really must have
online education due to the nature of their work. Of course
Hawaii is also geographically isolated meaning that the </i><i>range of educational choices is not what it is in other areas,
making online education particularly valuable to Hawaii
residents.</i><br />
<br />
<b>
Pavel: How do you react to comments that MOOC are undermining
higher education? For instance, some people are inclined to say
that if awarded with academic credit, they would have no
incentive to physically go to school when they can take
everything online for free or a symbolic fee? Do you think this
is a valid argument? Do you think this is a problem for faculty?</b>
<br />
<b><br /></b>
<i>Sam: I guess the argument is that as MOOCs start awarding academic
credit then many students might not attend a bricks and mortar
"academy", thus undermining higher education institutions? I
think it's entirely possible that MOOCs may undermine higher
education institutions that are not offering the highest quality
of courses and value for money to their students. I don't think
that anything that itself promotes education can undermine
education unless one posits that students receive some
particular benefit from attending a physical institution that
they cannot receive online. Individuals have been taking
distance and online courses around the world for many many
years. Here in the UK the Open University has been delivering
higher education without physical institutions for some 30 odd
years, and the UK still has a very strong set of higher
education institutions in the physical domain. I think it is an
open question as to the value that students receive from
attending a physical institution, and the most important thing
in a free market is that of choice. Students should have a free
choice as to where they look for support to achieve their
learning goals, and should not be paying over the odds for the
support they receive. I think the only challenge that MOOCs and
other educational technology developments present for faculty is
for those faculty who are not delivering the highest quality
educational experience possible, and for those institutions who
are charging over the odds.</i><br />
<br />
<b>
Pavel: Finally, is there anything that you want to add, or a
question that i should have asked you?</b>
<br />
<b><br /></b>
<i>
A general comment would be that globally I think we have
education back to front. Rather than asking what skills we
should be teaching students and what subjects they should be
learning, we should be asking our students what excites them,
what they want to learn and what they want to build.</i>Sam Josephhttp://www.blogger.com/profile/10788506730233381803noreply@blogger.com3tag:blogger.com,1999:blog-9148683707845127605.post-63078043924764072092013-05-24T09:14:00.002-07:002013-05-24T09:14:15.295-07:00Scheduling Remote Pair Programming SessionsSo maybe it's just me, but it feels like there has been a recent explosion in interest in remote pair programming. Maybe it's being going on for ages, but I recently discovered great resources such as Joe Moore's <a href="http://remotepairprogramming.com/">http://remotepairprogramming.com/</a> and <a href="http://www.pairprogramwith.me/">http://www.pairprogramwith.me/</a><br />
<br />
I've now added the PairWithMe badge to all my projects on Github, and I follow the #pairwithme hashtag on twitter, which is exposing me to things like this remote pair programming meetup group:<br />
<br />
<a href="http://www.meetup.com/remotepairprogrammers/">http://www.meetup.com/remotepairprogrammers/</a><br />
<br />
@batarski and @eee_c were saying on twitter that the #pairwithme website should coordinate and maybe host the sessions. I gave a lame tweet reply that that would be cool, and that some other sites were doing that, but didn't really have space to list the, so here they are:<br />
<br />
<ul>
<li><a href="http://pairwith.me/">http://pairwith.me/</a> Alpha - very basic</li>
<li><a href="http://rubypair.com/">http://rubypair.com/</a> Gamma? - looks cool</li>
<li><a href="http://tansaku.github.io/PairProgrammingScheduler/">http://tansaku.github.io/PairProgrammingScheduler/</a> Alpha - hardly functional - this is my attempt so far :-)</li>
<li><a href="https://www.virtualpairprogrammers.com/">https://www.virtualpairprogrammers.com/</a> Gamma? Paid site - training courses</li>
<li><a href="https://github.com/samnang/pair-with-me">https://github.com/samnang/pair-with-me</a> Alpha? Not functional any more?</li>
</ul>
<div>
And there was another one I signed up with recently, but I lost my Chrome history in a recent crash, so can't find it at the moment :-(</div>
<div>
<br /></div>
<div>
However although I've made contact with a few people through these (mainly just on twitter) I think I've only got a couple of remote pair programming sessions going from them. Far more intensive has been the now 30+ and counting hours I've been remote pair programming on our EdX SaaS <a href="https://github.com/tansaku/LocalSupport/">LocalSupport</a> project which we are mainly co-ordinating through Skype chat rooms. Here's a playlist of those videos</div>
<div>
<br /></div>
<div>
<a href="http://www.youtube.com/playlist?list=PLjbL0BCR04Q1-x5p5L5V11Vm58VlTgPzw">http://www.youtube.com/playlist?list=PLjbL0BCR04Q1-x5p5L5V11Vm58VlTgPzw</a></div>
<div>
<br /></div>
<div>
I'm looking forward to seeing more solutions to the challenge of remote pair programming scheduling.</div>
Sam Josephhttp://www.blogger.com/profile/10788506730233381803noreply@blogger.com3tag:blogger.com,1999:blog-9148683707845127605.post-27692064985469160822013-04-27T00:23:00.001-07:002013-04-27T00:38:17.290-07:00HCI Wireframing Assignment: Pair Programming SchedulerThese prototypes *<b>very*</b> loosely based on my earlier <a href="http://linklens.blogspot.co.uk/2013/04/coursera-hci-storyboarding-remote-pair.html">storyboards</a> - basically I switched away from interventions during actual pairing, to think about scheduling pairing sessions.<br />
<br />
Prototype 1 where the user indicates their availability first:<br />
<br />
<iframe height="400" src="https://dl.dropboxusercontent.com/u/13196858/PairProgrammingScheduler.pdf" width="600"></iframe><br />
<br />
<br />
Prototype 2 where the user browsers who is available first. (more like the <a href="http://pair-with-me.herokuapp.com/">http://pair-with-me.herokuapp.com/</a> tweet aggregator)<br />
<br />
<iframe height="400" src="https://dl.dropboxusercontent.com/u/13196858/PairProgrammingScheduler%202.pdf" width="600"></iframe>Sam Josephhttp://www.blogger.com/profile/10788506730233381803noreply@blogger.com0tag:blogger.com,1999:blog-9148683707845127605.post-30179762103945658522013-04-25T11:56:00.001-07:002013-04-26T06:15:55.903-07:00What I'd do with a ScreenHero APISo <a href="http://screenhero.com/">Screenhero</a> rocks, but I was thinking it would be really cool if it had an API so you could scaffold the interaction between pair programmers collaborating on code.<br />
<br />
Imagine that you've got a couple of relative novice programmers who are new to pair programming and to related concepts like TDD. Think about being able to have suggestions pop up once they had completed certain parts of the task. You could do this without plugging directly into ScreenHero, but what would be nice would be to ensure that one person did one portion of coding, followed by the other person doing another.<br />
<br />
I guess what I should really do first is work out the precise flow of a scaffolded pairing session and then think more carefully about what prompts the users would need, and how to keep track of who's doing the work. Ultimately we want to be able to have a reliable automated check that actual pair programming is happening rather than just one person doing all the work.<br />
<br />
Perhaps having each person check their code into github separately - but that then pulls the pair apart. I guess it raises the question of how to attribute code changes in a version repository when two people are working on the same file at the same time on one computer ...<br />
<br />
<div style="text-align: left;">
Follow up - just watched great talk by Sam Livingston-Gray, which mentioned a pair programming protocols "Ping Pong" and some variations called "Change the Message" and "one-undermanship":</div>
<div style="text-align: center;">
<br /></div>
<div style="text-align: center;">
<iframe allowfullscreen="" frameborder="0" height="315" src="http://www.youtube.com/embed/W_hsEi_UZHE" width="420"></iframe>
</div>
<div style="text-align: center;">
<br /></div>
<div style="text-align: left;">
Looks like ping pong protocol goes way back. I think this is what I'd really like to be scaffolding.</div>
<br />
<a href="http://c2.com/cgi/wiki?PairProgrammingPingPongPattern">http://c2.com/cgi/wiki?PairProgrammingPingPongPattern</a><br />
<br />
I wonder if the best way to scaffold would be through checkins to a Github repo - then at least you could check in retrospect how people coded ...Sam Josephhttp://www.blogger.com/profile/10788506730233381803noreply@blogger.com0tag:blogger.com,1999:blog-9148683707845127605.post-22825028767425971442013-04-25T05:22:00.001-07:002013-04-25T05:22:33.640-07:00More HCI BloopersSo I do appreciate the folks at Cengage Learning providing me online access to a textbook via the coursesmart system.<br />
<br />
I was less excited about getting locked out of my account for skimming quickly through the book. Then to unlock it there was a helpful chat window, but the multiply nested form I had to fill in to get into the chat itself was a great HCI blooper in my opinion. It took me three or four attempts to work out which category "account locked" should go under and in the end it was none of them, so I just wasted 3 extra minutes, and eventually chose some random sub-categories:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjIzfPbx5Qs6re2X0JKVd_1_wYAnEX_ZXpKtuJIIqmmwnjwQ3qHYRNCKXyvvGJl3e54tzxvY4t0XavqeMduP1RimZiER3vshbrwUqFv6ZxpZcVgDGHYK-hoaMG-dl-__igE8leuErnFMQ-c/s1600/AnnoyingSequenceOfFormsForCourseSmartHelpChat.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="330" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjIzfPbx5Qs6re2X0JKVd_1_wYAnEX_ZXpKtuJIIqmmwnjwQ3qHYRNCKXyvvGJl3e54tzxvY4t0XavqeMduP1RimZiER3vshbrwUqFv6ZxpZcVgDGHYK-hoaMG-dl-__igE8leuErnFMQ-c/s400/AnnoyingSequenceOfFormsForCourseSmartHelpChat.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
The guy in the chat did help me out pretty promptly, but didn't seem very interested in my suggestions regarding the lack of skimming functionality or the problems with the form. I did recommend that he direct their interface designers to the <a href="https://www.coursera.org/course/hci">free Stanford HCI course</a> but he didn't seem particularly interested ... :-)</div>
<br />Sam Josephhttp://www.blogger.com/profile/10788506730233381803noreply@blogger.com0tag:blogger.com,1999:blog-9148683707845127605.post-71094882836652370452013-04-22T06:38:00.001-07:002013-04-22T06:44:06.694-07:00Remote Pair Programming for the Masses<br />
<div class="MsoNormal">
<span lang="EN-US">I am submitting this project for funding from my University. It concerns creating an
online calendar in which students from around the world can register their
interest in remote pair programming.
Evidence is mounting that pair programming is the most effective way to
learn how to program, and also generates higher quality code faster. One of the key challenges is finding someone
to pair with; however an appropriately structured calendar that took into
account timezone differences, programming skill level etc., could create an
environment where there was always someone ready to pair program with you day
or night.</span></div>
<div class="MsoNormal">
<span lang="EN-US"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-US"><b>Project Description</b></span></div>
<div class="MsoNormal">
<span lang="EN-US"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-US"></span></div>
<div class="MsoNormal">
Pair programming is the process whereby two programmers work together on the same coding problem. Usually one will act as a navigator and the other as a driver with control of the keyboard. The central idea is that there is an immediate feedback loop whereby the navigator can check the code the driver creates and the two can discuss possible alternatives. This is similar to the kind of feedback that may come from code reviews and project retrospectives, but is on a much smaller timescale.</div>
<div class="MsoNormal">
Pair programming is being increasingly adopted in industry although there is a general resistance based on the fear productivity will be lower given that two programmers could be working on two different tasks. However Cockburn & Williams (2000) demonstrated the following significant benefits of pair programming:</div>
<div class="MsoNormal">
</div>
<ul>
<li>many mistakes get caught as they are being typed in rather than in QA test or in the field (continuous code reviews)</li>
<li>the end defect content is statistically lower (continuous code reviews)</li>
<li>the designs are better and code length shorter (ongoing brainstorming and pair relaying)</li>
<li>the team solves problems faster (pair relaying)</li>
<li>the people learn significantly more, about the system and about software development (line-of-sight learning)</li>
<li>the project ends up with multiple people understanding each piece of the system</li>
<li>the people learn to work together and talk more often together, giving better information flow and team dynamics</li>
<li>people enjoy their work more</li>
</ul>
<br />
<div class="MsoNormal">
Cockburn & Williams go on to point out that the development cost for these benefits is not the 100% that might be expected, but is around 15%; which is quickly paid back for in shorter and less expensive testing, quality assurance, and field support. Of particular note are the learning benefits, which have also been noted by Williams & Kessler (2003) who indicate that:</div>
<blockquote class="tr_bq">
"<i>Knowledge is constantly being passed between partners, from tool usage tips to design and programming idioms. The partners take turns being the teacher and the student. Even unspoken skills and habits cross partners</i>."</blockquote>
<div class="MsoNormal">
Canfora et al (2007) found a reduction in defect rates of 15% to 50%, varying depending on programmer experience and task complexity, while a number of other studies found similar improvements (Lui, 2006; Arisholm et al., 2007; Lui et al., 2008), however meta-analyses have cautioned that this all depends on a number of factors and that you cannot expect “faster and better and cheaper” (Hannay et al., 2009).</div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Independently of whether pair programming is more effective than solo programming, the educational benefits seem clear. McDowell et al (2002) and Salleh et al. (2011) both found that students’ grades and satisfaction improved with pair programming as compared to programming solo. Furthermore McDowell et al. (2002) saw increases in retention indicating that the use of pair programming could prevent students from dropping out.</div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Schümmer & Lukosch (2009) describe some of the additional difficulties faced when pair programming remotely, and there are also concerns that productivity and satisfaction can drop during novice-novice pairing is used without sufficient availability of a mentor for coaching (Stephens & Rosenberg, 2003). All of this indicates that there is a huge pedagogical potential for remote pair programming, but that the process needs to be carefully scaffolded to ensure successful outcomes.</div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
</div>
<div class="MsoNormal">
<b>Specific Aims</b></div>
<div class="MsoNormal">
</div>
<ul>
<li>Create online calendar for remote pair programming (RPP)</li>
<li>Provide support for editing, discovery and matchmaking</li>
<li>Scaffolding successful RPP sessions</li>
<li>Recording RPP sessions for analysis</li>
<li>Using results of analysis to improve the RPP experience</li>
</ul>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b>Significance</b></div>
<div class="MsoNormal">
<b><br /></b></div>
<div class="MsoNormal">
The significance of this work is that it is becoming clear that pair programming is the fastest and most effective way to learn how to program, and yet in general it can be very challenging to find pair programming opportunities. A well organized calendar that allowed students from around the world to discover other students available to pair program could be invaluable in increasing student satisfaction and minimizing the amount of time required to become a confident programmer.</div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b>Work so far</b></div>
<div class="MsoNormal">
<b><br /></b></div>
<div class="MsoNormal">
</div>
We have created a <a href="https://sites.google.com/site/saasellsprojects/pair-programming-calendar">shared Google calendar</a> and <a href="https://sites.google.com/site/saasellsprojects/pair-programming-form">sign up form</a> that now has 30 students signed up. Remote pair programming sessions have been recorded with at least 10 of these students and a <a href="http://linklens.blogspot.co.uk/2013/04/coursera-hci-needfinding-remote-pair.html">preliminary analysis of one session</a> has been conducted.<br />
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
</div>
<div class="MsoNormal">
<b>Bibliography</b></div>
<div class="MsoNormal">
</div>
<ul>
<li>Arisholm, Erik; Hans Gallis, Tore Dybå, Dag I.K. Sjøberg (February 2007). "Evaluating Pair Programming with Respect to System Complexity and Programmer Expertise". IEEE Transactions on Software Engineering 33 (2): 65–86.</li>
<li>Canfora, Gerardo; Aniello Cimitile, Felix Garcia, Mario Piattini, Corrado Aaron Visaggio (2007). "Evaluating performances of pair designing in industry". The Journal of Systems and Software 80 (80): 1317–1327.</li>
<li>Cockburn, A. & Williams, L. (2000). "The Costs and Benefits of Pair Programming". Proceedings of the First International Conference on Extreme Programming and Flexible Processes in Software Engineering (XP2000).</li>
<li>Hannay, J. E.; Dybå T., Arisholm E., Sjøberg D. I. K. (2009). "The Effectiveness of Pair Programming: A Meta-Analysis". Information and Software Technology 51 (7): 1110–1122.</li>
<li>Lui, K. M. (2006). "Pair programming productivity: Novice-novice vs. expert-expert". International Journal of Human-Computer Studies 64 (9): 915–925</li>
<li>Lui, K. M., Chan K. C. C., Nosek J. T. (2008). "The Effect of Pairs in Program Design Tasks". IEEE Transactions on Software Engineering 32 (2): 197–211.</li>
<li>McDowell, C., Werner, L., Bullock, H., & Fernald, J. (2002). The effects of pair-programming on performance in an introductory programming course. ACM SIGCSE Bulletin, 34(1), 38-42.</li>
<li>Salleh, N., Mendes, E., Grundy, J.C. (2011) Empirical Studies of Pair Programming for CS/SE Teaching in Higher Education: A Systematic Literature Review. IEEE Trans. Software Eng. 509-525.</li>
<li>Schümmer, T. & Lukosch S. (2009) "Understanding Tools and Practices for Distributed Pair Programming". Journal of Universal Computer Science 15 (16): 3101–3125.</li>
<li>Stephens M. & Rosenberg D. "Extreme Programming Refactored: The Case Against XP", Berkeley, CA: Apress, 2003.</li>
<li>Williams, Laurie; Kessler, Robert (2003). Pair Programming Illuminated. Addison-Wesley. ISBN 0-201-74576-3.</li>
</ul>
<br />
<div>
<br /></div>
<br />
<div>
<br /></div>
<br />Sam Josephhttp://www.blogger.com/profile/10788506730233381803noreply@blogger.com0tag:blogger.com,1999:blog-9148683707845127605.post-25797244030952285812013-04-19T10:06:00.000-07:002013-04-19T23:33:15.731-07:00Coursera HCI StoryBoarding: Remote Pair Programming<b>List five existing designs (inspirations) that relate to your thinking. The relationship could be very concrete or very abstract. </b><br />
<br />
1. Joe Moore's Blog: <a href="http://remotepairprogramming.com/">http://remotepairprogramming.com/</a> Joe collects lots of really useful tips on remote pair programming in his tumblr blog.<br />
<br />
2. Android intents which allow android apps to interact with each other seemlessly. This allows re-use of components of android apps.<br />
<br />
3. Growl notifications on OSX - which pop and then fade away. A wonderful lightweight notification system to keep you in the loop with minimum annoyance.<br />
<br />
4. CmdShift4 screen capture on OSX - allows you to take a snapshot of some portion of the screen - not very intuitive chord key (but are they ever), but just a dream to use - select a square on the screen and image gets dropped to desktop - even better when you set the drop point to be in your dropbox public folder, since then you instantly have a publicly shareable URL to share the screen shot in text chat or email etc.<br />
<br />
5. Automated centering guides that XCode's GUI builder (and now Google Presenetations) uses to help you drop a picture or GUI element into alignment with surrounding elements. Fantastically intuitive and easy to use - they just appear and help as you go about doing what you were already doing.<br />
<div>
<br /></div>
<b>Which of the design briefs does your observation (from Assignment 1) fall under: Change, Glance, or Time? Explain what activity you observed and how it relates to the brief. Write down your Point of View.</b><br />
<br />
Change. I observed programmers remote pair (triple programming) which they are doing in order to solve programming problems, but also to improve their programming skills.<br />
<br />
<br />
<b>Upload a comprehensive set of digital photos or scans of both of your storyboards.</b><br />
Point of view: Remote Pair Programming should be as natural and easy as being in the same room with other programmers, just even more convenient.<br />
<div>
<br /></div>
<div>
<br /></div>
<div>
<div style="color: #333333; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 21px; margin-bottom: 10.5px;">
1. Multiple cursors for different users each with colour and/or logo</div>
<div style="color: #333333; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 21px; margin-bottom: 10.5px;">
<br />
<img alt="" data-mce-src="https://coursera-uploads.s3.amazonaws.com/user-a08553ebf5d9e0bd9965d3ff/970447/asst-19/970447-516ecf1fafff22.76579908.png" src="https://coursera-uploads.s3.amazonaws.com/user-a08553ebf5d9e0bd9965d3ff/970447/asst-19/970447-516ecf1fafff22.76579908.png" style="border: 0px; cursor: default; height: auto; max-width: 100%; vertical-align: middle;" /></div>
<div style="color: #333333; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 21px; margin-bottom: 10.5px;">
<br /></div>
<div style="color: #333333; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 21px; margin-bottom: 10.5px;">
2. Always front overlaid transparent chat</div>
<div style="color: #333333; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 21px; margin-bottom: 10.5px;">
<br />
<img alt="" data-mce-src="https://coursera-uploads.s3.amazonaws.com/user-a08553ebf5d9e0bd9965d3ff/970447/asst-19/970447-5171763fdf9f40.68334456.png" src="https://coursera-uploads.s3.amazonaws.com/user-a08553ebf5d9e0bd9965d3ff/970447/asst-19/970447-5171763fdf9f40.68334456.png" style="border: 0px; cursor: default; height: auto; max-width: 100%; vertical-align: middle;" /></div>
</div>
Sam Josephhttp://www.blogger.com/profile/10788506730233381803noreply@blogger.com0tag:blogger.com,1999:blog-9148683707845127605.post-14646425579945437132013-04-19T09:57:00.002-07:002013-04-19T10:01:27.068-07:00Coursera HCI Needfinding: Remote Pair ProgrammingScott Klemmer's HCI class is great, but I think one thing that would make it even better would be if all the students design submissions were public on blogs for all to share and see (assuming student consent). I love to share, so here's my assignment 1:<br />
<br />
<br />
<b>Design Brief: Change</b><br />
<br />
I observed groups of programmers programming together, trying to improve their programming skills and change themselves from less experienced programmers into more experienced programmers.<br />
<br />
<br />
<b>Describe how three people performed an activity related to the brief you chose. In particular, you should focus on a particularly interesting moment/breakdown/workaround from the observation. Upload a photograph or sketch of each person performing the described activity, with a short caption.</b><br />
<br />
Three programmers worked on a particular feature from a programming project. For reference here is the complete video: <a href="https://www.youtube.com/watch?v=ELNZTymLSaE&feature=player_embedded" style="box-sizing: border-box; color: #2e67b1; text-decoration: none;">https://www.youtube.com/watch?v=ELNZTymLSaE&feature=player_embedded</a> I obtained permission from all programmers to share the session<br />
<div style="box-sizing: border-box; font-size: 14px; margin-bottom: 10.5px;">
<br /></div>
<div style="box-sizing: border-box; font-size: 14px; margin-bottom: 10.5px;">
Interesting moments/breakdowns/workarounds</div>
<div style="box-sizing: border-box; font-size: 14px; margin-bottom: 10.5px;">
1: microphone not working for one remote programmer and that programmer used skype text chat to stay in touch [video time 10:50, 22:03, 41:22 51:38, 1:08:48, 1:22:00]</div>
<div style="box-sizing: border-box; font-size: 14px; margin-bottom: 10.5px;">
<img alt="" height="343" src="https://coursera-uploads.s3.amazonaws.com/user-a08553ebf5d9e0bd9965d3ff/970447/asst-15/970447-516869719add75.12131750.png" style="border: 0px; box-sizing: border-box; height: auto; max-width: 100%; vertical-align: middle;" width="400" /></div>
<div style="box-sizing: border-box; font-size: 14px; margin-bottom: 10.5px;">
<br /></div>
<div style="box-sizing: border-box; font-size: 14px; margin-bottom: 10.5px;">
2. One user learns from the other about a coding abbreviaton ("rails c" for "rails console") [video time 13:30]</div>
<div style="box-sizing: border-box; font-size: 14px; margin-bottom: 10.5px;">
<img alt="" height="222" src="https://coursera-uploads.s3.amazonaws.com/user-a08553ebf5d9e0bd9965d3ff/970447/asst-15/970447-516b16e9f387f0.39093520.png" style="border: 0px; box-sizing: border-box; height: auto; max-width: 100%; vertical-align: middle;" width="400" /></div>
<div style="box-sizing: border-box; font-size: 14px; margin-bottom: 10.5px;">
<br /></div>
<div style="box-sizing: border-box; font-size: 14px; margin-bottom: 10.5px;">
3. Use of reference to line numbers in text editor to help focus on particular things in the session [video time 49:56, also at 58:28]</div>
<div style="box-sizing: border-box; font-size: 14px; margin-bottom: 10.5px;">
<img alt="" height="327" src="https://coursera-uploads.s3.amazonaws.com/user-a08553ebf5d9e0bd9965d3ff/970447/asst-15/970447-516b17666563c5.19050700.png" style="border: 0px; box-sizing: border-box; height: auto; max-width: 100%; vertical-align: middle;" width="400" /></div>
<div style="box-sizing: border-box; font-size: 14px; margin-bottom: 10.5px;">
<br /></div>
<div style="box-sizing: border-box; font-size: 14px; margin-bottom: 10.5px;">
Here are my detailed notes on the actions of the participants, M, J, and S:</div>
<div style="box-sizing: border-box; font-size: 14px; margin-bottom: 10.5px;">
<br /></div>
<br /><br />3:30 working out how to add an epic in pivotal tracker<br /><br />6:37 J using putty for some reason<br /><br />8:44 J zooming size of screen (discussion of use of vi in the background)<br /><br />10:50 S using Skype chat because his mic is not working, he can hear the other two, but they can't hear him<br /><br />13:30 inspecting models from database in rails console, J types "rails c" which M had not known was contraction for "rails console"<br /><br />14:55 J uses highlighting to point out link with the token for the login<br /><br />15:36 J already did some test install based on link S sent? was an older version, lead to some errors but then J followed some more instructions on the devise website?<br /><br />17:18 chrome gets stuck, but no big deal<br /><br />19:36 M wonders where she is - navigating pivotal tracker<br /><br />20:40 generating feature description? should this be coming from client<br /><br />22:03 more text chat from S<br /><br />26:23 M confirms the "so that" part of the feature is basically a comment<br /><br />29:12 M thought that some menus were in spanish, but J says no, but they moved on<br /><br />32:15 comparing tracker numbers in feature - wish we had links - there was confusion about which story they were working on<br /><br />35:15 M interested in tab completion<br /><br />35:45 M couldn't see the terminal at one point. J moved it down<br /><br />37:52 J playing with zoom again<br /><br />40:32 M suggests password field and then need for pipe symbol<br /><br />41:22 Skype chat popup lets M know that S is texting to indicate 8 character restriction an be changed. Just after another popup indicates chatting on Skype in an unrelated discussion --> would be good if one could get a general heads up on the relevant chat …<br /><br />44:36 M suggests changing Then to When in feature<br /><br />~46:30 J asks for confirmation that they have finished everything they need in the scenario<br /><br />49:28 discussing whether the user name is in the system, or only the email<br /><br />49:56 M pointed out indent issues to J used line numbers to indicate where to look<br /><br />50:48 M unable to hear J, asked for confirmation- he was checking again if they'd finished<br /><br />51:38 S reminds them of losing password feature by text chat M tells J via audio<br /><br />53:20 work out that they need to add another user story<br /><br />55:00 J accidentally? clicks on Skype and then needs to minimise<br /><br />56:28 M helps with spelling correction<br /><br />58:28 M says to J "you have a typo on line 37" again using line numbers<br /><br />59:15 M suggests a grammatical correction<br /><br />1:00:20 M suggests sign in page rather than log in page for consistency<br /><br />1:02:30 M has funny text message sound<br /><br />1:06:51 - more tips from S in Skype<br /><br />1:07:43 M corrects indentation<br /><br />1:08:48 S points out cucumber doesn't require the indentation, and M acknowledges but points out it is important that it should be consistent<br /><br />1:10:45 M points out some spelling mistakes<br /><br />1:14:00 J has to nip off to answer phone, M pauses the screen recorder and then restarts - perhaps we lost something?<br /><br />1:17:00 can hear Skype notification sounds(?) in the background<br /><br />1:18:40 Other Skype users coming online notifications<br /><br />1:20:29 J says he needs to finish up<br /><br />1:21:00 discussion about how M can access the work that J has done on his computer<br /><br />1:22:00 S points out that changes can be pushed to github repo, and there is discussion about pushing that to branch<br /><br />1:23:50 saying yes in text to S, but he can hear them no?<br />
<br />
<br />
<b>Write a list of needs/goals/tasks inspired by what you observed</b><br />
<br />
1. text chat could be spoken aloud by some automated text to speech system to allow those without microphones to participate<br />
<br />
2. would be good to have each user have their own mouse so that they can point to things in the shared space<br />
<br />
3. Would be good if the only relevant text chat interjected<br />
<br />
4. Would be good if the text chat could appear without the users having to switch windows <br />
<br />
5. Would be nice if the text editor caught some spelling mistakes in cucumber feature files<br />
<br />
6. Would be good it important external calls could pause the screen recorder<br />
<br />
7. Would be good if paused screen recorder had more observable "pause" status<br />
<br />
8. Would be good if users could see more of each others status in a lightweight way, e.g. whether they are busy, distracted etc.<br />
<br />
9. Would be good if all users could edit in parallel, e.g. fixing each others spelling mistakes<br />
<br />
10. shared whiteboard for sketching would be good<br />
<br />
11. bigger screen real estate for all would be good, e.g. at least two monitors for everyone and everyone able to see all of each others screens, so more like sharing an entire wall<br />
<br />
12. would be good if the task being attempted was scaffolding in some way - could tick off when certain goals had been achieved<br />
<br />
13. would be good if sharing and recording were always on, so that it was just like dropping into a room rather than having to set up a load of software<br />
<br />
14. would be good to have some way to have other groups of individuals exist in the same space so that they could be called over to help on particular issues and then drift off<br />
<br />
15. every increase in bandwidth to support higher resolution video and audio streaming would be useful for the participants<br />
<br />
16. everyone could use help finding shared times for pair programming, coordinating the agenda, preparing to make the best use of the session<br />
<br />
<br />
<br />Sam Josephhttp://www.blogger.com/profile/10788506730233381803noreply@blogger.com1tag:blogger.com,1999:blog-9148683707845127605.post-38351778424785863122013-04-14T08:11:00.003-07:002013-04-14T08:11:46.435-07:00Combining Remote Pair Programming & Massively Open Online Classes (MOOCs)I'm getting very excited about the educational potential of remote pair programming. We've been trying out remote pair programming on our <a href="http://github.com/tansaku/LocalSupport">LocalSupport</a> project as part of the <a href="http://beta.saasbook.info/">EdX Software as a Service</a> online course.<br />
<br />
The debate about when to use pair programming in real world projects still rages, but from my personal experience it's a huge win almost every time from an educational perspective in terms of programmers learning new languages, techniques and skills. <br />
<br />
However one of the challenges is finding someone to pair program with. To that end we created a shared Google Calendar that includes the times and contact details of everyone wanting to be pair program.<br />
<br />
<iframe height="400" src="https://www.google.com/calendar/embed?src=shs6gf6uf0flntkl1fcvicea4o%40group.calendar.google.com&mode=week" width="100%"></iframe>
<br />
<br />
Programmers who want to join the calendar fill out the following form:<br />
<br />
<a href="https://sites.google.com/site/saasellsprojects/pair-programming-form">https://sites.google.com/site/saasellsprojects/pair-programming-form</a><br />
<br />
and at the moment we then add them by hand to the calendar, as well as giving them edit permissions to allow them to change their availability over time. Naturally in the long run we'd like to totally automate this process.<br />
<br />
So far we've had over 10 successful remote pair programming sessions with from 2 to 5 programmers participating in each. You can see recordings of those sessions here:<br />
<br />
<a href="https://sites.google.com/site/saasellsprojects/projects/local-support/pair-programming">https://sites.google.com/site/saasellsprojects/projects/local-support/pair-programming</a><br />
<br />
What we're also finding is a split between those relatively new to the Rails stack we are working on (mainly EdX 169.1X students) and those with more experience (those who have completed EdX 169.2X). The latter now tend to be working on our LocalSupport non-profit project, while the former are starting to remote pair with each other to work on simple ruby and rails assignments.<br />
<br />
I've also started to discover professionals like <a href="http://remotepairprogramming.com/">Joe Moore</a> who have been remote pair programming for a number of years, and am discovering that there are a lot more options beyond the Skype and Google Hangouts we've been using so far.<br />
<br />
What's really exciting here is the combination of the Massively Open Online Classes (MOOCs) with the up close and personal feedback of pair programming sessions. A very powerful learning combination in my opinion. Just navigating all the libraries in the rails stacks and getting used to git pull requests etc. is the kind of thing that pair programming really helps for. Lots of relatively confident learners can get put off when they are stuck alone battling against these things.<br />
<br />
If we're lucky this kind of calendar for pair programming opportunities can be embedded into MOOCs and scaled so that any time of day or night you'll always be able to find someone to pair with, and the learning opportunities will be endless ...Sam Josephhttp://www.blogger.com/profile/10788506730233381803noreply@blogger.com0tag:blogger.com,1999:blog-9148683707845127605.post-61605358001099397402013-03-22T12:29:00.002-07:002013-03-22T12:44:20.810-07:00Benefit of Quizzes for LearningIn a recent discussion I was pointed to <a href="http://www.nytimes.com/2011/01/21/science/21memory.html?src=me&ref=homepage">this article on the benefits of testing</a> as evidence that quizzes have pedagogical value. Given that memory researchers have been getting results that testing students improved their retention for many years I was a little surprised that the NYT was presenting these results as quite so novel. For example Dempster (1996)
provides multiple citations of this effect going back as early as the
1920s. I guess that's journalism for you :-) However the study being reported showed that simple testing beat drawing concept maps which is very interesting, although I think the results might be
reversed if the students were practising generating concept maps from
memory in the concept map condition.<br />
<br />
This is an area that I have researched in some detail, particularly as regards vocabulary learning:<br />
<blockquote class="tr_bq">
Joseph S.R.H., Watanabe Y., Shiung Y.-J., Choi B. & Robbins C. (2009) <a href="http://apsce.net/RPTEL/2009_04_02_2.PDF">Key Aspects of Computer Assisted Vocabulary Learning (CAVL): Combined Effects of Media, Sequencing and Task Type</a>. Research and Practice in Technology Enhanced Learning.<br />
4(2) 1-36.</blockquote>
I certainly don't doubt that retrieval practise promotes retention, or more simple that taking tests helps you remember things. I think the problem with the study mentioned in the NYT and those
referred to by Dempster is that what they are really showing is that taking a test is
great preperation for doing well on tests. If the objective is to have
students pass tests then yes, let them do lots of tests.<br />
<br />
A more important question is what are the target skills that we are
hoping that students acquire? For example in my programming classes
I am hoping that my students will learn to program. So should I be
setting them written tests, or should I be setting them programming
assignments?<br />
<br />
In my HCI classes I want my students to learn design skills, to produce
good designs, so should I set them written tests on the subject of
design, or should I have them do design assignments?<br />
<br />
<a href="http://hci-class.org/">Scott Klemmer's HCI classes</a> have lots of great assignments in which students practise design
skills, as well as standalone and media integrated quizzes. I don't think there is any doubt as to value of the assignments, but are quizzes the best way to get the students to the
point that they can practise the target skills? Simple quizzes built
into the lecture material seem relatively benign, and these are common place throughout the current slew of MOOC courses, and are arguably good at keeping student focus on the short videos provided. I have also been impressed with the use of quizzes with voting and discussion as seen in the <a href="http://www.youtube.com/course?list=EC-XXv-cvA_iDTKE56ZRv92RJNnLmy2aZh">Berkeley Software Engineering classes</a>. However for students in MOOC land they are watching the videos in a variety of settings and I wonder if the short quizzes are really worth the effort.<br />
<br />
Furthermore I think longer harder
quizzes such as those in the Berkeley Software Engineering course (both face to face and online) can be rather intimidating for
many students. And even for confident students, quizzes can be fundamentally a boring
activity. Of course quizzes give instructors insight into student progress, but they are also potentially demotivating for students and one might reasonably ask if they have any effect on the ability of the students to perform the relevant target skills?<br />
<br />
Given that we are considering courses that will provide at least some material in text and video and
other media, the key question to me is whether quizzes integrated into the media are directly beneficial in terms of subsequent
performance on target skills?<br />
<br />
To operationalize all this we might try to device a study to test if is more
valuable for a student to be spending it absorbing material passively
and then taking a short quiz, or absorbing more material passively, or
absorbing it passively and than doing an active assignment, or perhaps
just doing an active assignment with the
whole of that time period? What might our experimental conditions look like? These perhaps:<br />
<ul>
<li>10 mins passive material, 5 mins on quiz</li>
<li>15 mins on passive material</li>
<li>10 mins passive material, 5 mins on active relevant assignment</li>
<li>15 mins on active relevant assignment</li>
</ul>
And by a relevant assignment, I mean ideally something that is relevant to that individual student, such as a project that they are actually motivated to work on. It seems like there is a great opportunity for AB testing in the current MOOCs - give a randomly selected half of the students no quizzes during the lectures, and the other half the integrated quizzes, and compare the quality of outcomes. Or three conditions perhaps:<br />
<br />
A) No Quizzes<br />
B) Simple Quizzes<br />
C) Harder Quizzes<br />
<br />
Maybe given some chunk of time, say 15 minutes, it's unrealistic to have every student work on an active relevant assignment - maybe given 15 minutes at the end of the day, all people can manage is some passive material and then a quick quiz to help them absorb it?<br />
<br />
On a more ethereal note I start to wonder if it is ever very effective to provide information to a student if they haven't expressed a need for it, or at least an interest in it. For most effective learning perhaps we need to manuever the student (or perhaps ourselves?) into a position where they
ask a question of their own violition and then respond to them with an answer
and perhaps a further question?<br />
<br />
<br />
<br />
Dempster, F. N. (1996). Distributing and managing the conditions of encoding and practice. In<br />
R. Bjork, & E. Bjork (Eds.), Memory (pp. 317–344). San Diego, CA: Academic PressSam Josephhttp://www.blogger.com/profile/10788506730233381803noreply@blogger.com0tag:blogger.com,1999:blog-9148683707845127605.post-83667825375843731912013-03-12T05:45:00.001-07:002013-03-12T05:48:09.735-07:00HyperLocal News in JustInMindBuilding on my earlier <a href="http://linklens.blogspot.co.uk/2013/02/interactive-hyperlocal-news-mockup.html">Balsamiq prototype</a> here's another version of the HyperLocalNews app using the <a href="https://www.justinmind.com/">JustInMind</a> framework. This gives a much higher fidelity prototype which to the untrained eye might look more like the real thing. The danger here is that a client might think you've already built the entire app :-)<br />
<br />
Try clicking on the map markers:<br />
<iframe height="800" src="https://www.justinmind.com/usernote/tests/10588812/10588815/10588817/index.html" width="460"></src></iframe><br />
<br />
Many thanks to creative commons sharing folk for the images to make the news stories more interesting:<br />
<br />
<a href="http://www.flickr.com/photos/7700821@N06/476498157/">http://www.flickr.com/photos/7700821@N06/476498157/</a><br />
<a href="http://www.flickr.com/photos/danshouse/137241182/">http://www.flickr.com/photos/danshouse/137241182/</a><br />
<a href="http://www.flickr.com/photos/us-pacific-command/7658198802/">http://www.flickr.com/photos/us-pacific-command/7658198802/</a><br />
<br />
Note that this is different from the Balsamiq exported PDF in that it's all HTML, and there's no accidental scrolling up and down. Getting a good look and feel of course relies on one knowing where to position things as they would appear on the appropriate smartphone.<br />
<br />Sam Josephhttp://www.blogger.com/profile/10788506730233381803noreply@blogger.com2tag:blogger.com,1999:blog-9148683707845127605.post-68026179000922555382013-03-11T06:49:00.000-07:002013-03-11T07:08:22.735-07:00Fest Swing Testing in Netbeans IDE 7.3I wanted to see if I could get the Fest Swing GUI testing framework running in NetBeans IDE (7.3). The <a href="http://plugins.netbeans.org/plugin/7964/fest-swing-plugin">existing plugin</a> was designed for 6.1; and some of the instructional images were missing. Once I'd worked out <a href="http://wiki.netbeans.org/InstallingAPlugin">how to install Netbeans IDE plugins</a> I found it did work with 7.3. Here's my version getting the sample code running in NetBeans IDE 7.3 beta 2<br />
<br />
1. Create a File --> New Project<br />
<br />
2. Choose Samples --> Java --> Anagram Game<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg76zl4Us2pgXeVgbfXOSq1FlmjTgnp3naT3g6d98le70Of2kiIwEhON4v_80OAS4unxd-fi2_knTW4pPo7BqdEgL2nfcqO6oYo3eq4P1DDFbUfh4DQn0jiCrox4RJNLbRebnDjPtyJHYYq/s1600/Screen+Shot+2013-03-11+at+13.32.23.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="227" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg76zl4Us2pgXeVgbfXOSq1FlmjTgnp3naT3g6d98le70Of2kiIwEhON4v_80OAS4unxd-fi2_knTW4pPo7BqdEgL2nfcqO6oYo3eq4P1DDFbUfh4DQn0jiCrox4RJNLbRebnDjPtyJHYYq/s320/Screen+Shot+2013-03-11+at+13.32.23.png" width="320" /></a><br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
3. Right click "Test Libraries" and choose Fest and JUnit 4.10<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhebLYWywKhGA-8YCYbdvgQGKPR0sHNZzVauCFhCBWDlpH36S7tvGpEoC9pnMFjSIWBMdp77SRTGoYl_UHZcyzE9yuFA6cduh7sA4AFMdNG84HWdGCl4dQ9bT6-epxcsjB67ZWgF1ui3iX2/s1600/Screen+Shot+2013-03-11+at+13.34.12.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="232" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhebLYWywKhGA-8YCYbdvgQGKPR0sHNZzVauCFhCBWDlpH36S7tvGpEoC9pnMFjSIWBMdp77SRTGoYl_UHZcyzE9yuFA6cduh7sA4AFMdNG84HWdGCl4dQ9bT6-epxcsjB67ZWgF1ui3iX2/s320/Screen+Shot+2013-03-11+at+13.34.12.png" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiSLm3mwuq7ZHHELe9SSQWFaWvciLmtZeOZhyphenhyphenlozpMUB6fOoDpraEatWmSSEmfChKmVrkS-nzfQ7xZj3IoAaY1MVAtqYrvizOqnUb-Ac8KruqAIMixq6JM-g0W4mWaMKT6Pc5xTkD_4e-Y_/s1600/Screen+Shot+2013-03-11+at+13.34.31.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="235" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiSLm3mwuq7ZHHELe9SSQWFaWvciLmtZeOZhyphenhyphenlozpMUB6fOoDpraEatWmSSEmfChKmVrkS-nzfQ7xZj3IoAaY1MVAtqYrvizOqnUb-Ac8KruqAIMixq6JM-g0W4mWaMKT6Pc5xTkD_4e-Y_/s320/Screen+Shot+2013-03-11+at+13.34.31.png" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
4. Create a File --> New File, and select Fest Test from Swing GUI forms:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgrZxmCTXCF1pTLJ1GO9G816JczriKFeGfnyynoiuPqXiOZb25v6MWUqZjUdP_VX9CeUdVnGHIuVMZUS97bApc6L_h79rKOtlN3K0-0JGMnMiwe4rozihW91GDE9nNI7N4jp3C_FDskUqmE/s1600/Screen+Shot+2013-03-11+at+13.37.23.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="228" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgrZxmCTXCF1pTLJ1GO9G816JczriKFeGfnyynoiuPqXiOZb25v6MWUqZjUdP_VX9CeUdVnGHIuVMZUS97bApc6L_h79rKOtlN3K0-0JGMnMiwe4rozihW91GDE9nNI7N4jp3C_FDskUqmE/s320/Screen+Shot+2013-03-11+at+13.37.23.png" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
5. Give it an appropriate name and put it in the com.toy.anagrams.ui package<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgsTxWlJ8pTPlH6d1aTOEwLc6P6Le1C7J7lP5-aedd8-Ro1EwjX9GhS5nxP4QbGcRz4Ah6-lqh7y7HiJKnMzo7ssBvGH3yT_Mu3fT4H6fbI3TRYb9CNM5RcOhPTnV2eT_oe_Xxl3hHAsvGk/s1600/Screen+Shot+2013-03-11+at+13.38.49.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="223" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgsTxWlJ8pTPlH6d1aTOEwLc6P6Le1C7J7lP5-aedd8-Ro1EwjX9GhS5nxP4QbGcRz4Ah6-lqh7y7HiJKnMzo7ssBvGH3yT_Mu3fT4H6fbI3TRYb9CNM5RcOhPTnV2eT_oe_Xxl3hHAsvGk/s320/Screen+Shot+2013-03-11+at+13.38.49.png" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
6. Add these three lines after the initComponents() method call in the constructor in Anagrams.java<br />
<br />
<span style="font-family: Courier New, Courier, monospace;"> guessedWord.setName("anagramField");</span><br />
<span style="font-family: Courier New, Courier, monospace;"> guessButton.setName("guessButton");</span><br />
<span style="font-family: Courier New, Courier, monospace;"> feedbackLabel.setName("resultLabel");</span><br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhAHHWIeBWkeAghqNlbEn5wdW0Lei_56pn91q6DeT7oHQMiwiopbRt_hgeeu2W-rp_Mh5u6Jvdc1iuDUWlPEHvjhXLl7K3h_opt4NozRpnvjLbfTp_SHdvbDk1dp3rXoQ4IwClYcHeGpzmr/s1600/Screen+Shot+2013-03-11+at+13.40.58.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="307" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhAHHWIeBWkeAghqNlbEn5wdW0Lei_56pn91q6DeT7oHQMiwiopbRt_hgeeu2W-rp_Mh5u6Jvdc1iuDUWlPEHvjhXLl7K3h_opt4NozRpnvjLbfTp_SHdvbDk1dp3rXoQ4IwClYcHeGpzmr/s640/Screen+Shot+2013-03-11+at+13.40.58.png" width="640" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
7. Uncomment line 26 in the fest file you created:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBRlq_QO7Mac0X-4EQ0zcukR5Hzbx9k6920QtJBPCLfLSnSpfZap3GwdkxGcmMdkTNAykk2VCSwssbdGDp2FIiMTwFacMcAlHKbA5zuQopgiEyuT8FL1u9yVLl-Fu3RAe86UxlEUmR5Sbv/s1600/Screen+Shot+2013-03-11+at+13.42.30.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBRlq_QO7Mac0X-4EQ0zcukR5Hzbx9k6920QtJBPCLfLSnSpfZap3GwdkxGcmMdkTNAykk2VCSwssbdGDp2FIiMTwFacMcAlHKbA5zuQopgiEyuT8FL1u9yVLl-Fu3RAe86UxlEUmR5Sbv/s640/Screen+Shot+2013-03-11+at+13.42.30.png" width="640" /></a></div>
<br />
<br />
8. Execute the fest file as a test, and watch the Anagram game be have text filled in automatically<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEha8Ph2bZ45qY8dFPH15g7O6pPQsB9QhaolzwrL-q73jwSQ530a5FK58fwcDCPv2d8stx_f6473mCIA1ddxoJ9hoZ3oJNHDFAhPtPNjy3z-ou4JTSBt3EXMdDfecukvk4qnPDYL9FVtPjjw/s1600/Screen+Shot+2013-03-11+at+13.43.49.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="175" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEha8Ph2bZ45qY8dFPH15g7O6pPQsB9QhaolzwrL-q73jwSQ530a5FK58fwcDCPv2d8stx_f6473mCIA1ddxoJ9hoZ3oJNHDFAhPtPNjy3z-ou4JTSBt3EXMdDfecukvk4qnPDYL9FVtPjjw/s200/Screen+Shot+2013-03-11+at+13.43.49.png" width="200" /></a></div>
Sam Josephhttp://www.blogger.com/profile/10788506730233381803noreply@blogger.com0tag:blogger.com,1999:blog-9148683707845127605.post-3551980826567537862013-02-26T03:29:00.000-08:002013-02-26T03:30:35.169-08:00Interactive HyperLocal News MockupBalsamiq is a great tool for quickly mocking up an interface, and you can even add interactivity, and my embedding a PDF export, you can have an interactive mockup running in your browser.
<br/><br/>
Try clicking on the map pins in the PDF below:
<br/><br/>
<embed src="https://dl.dropbox.com/u/13196858/Spr2013Mobile/HyperlocalNews.pdf" width="600" height="800">Sam Josephhttp://www.blogger.com/profile/10788506730233381803noreply@blogger.com0tag:blogger.com,1999:blog-9148683707845127605.post-29139265413724918782013-02-25T07:25:00.001-08:002013-02-25T07:25:05.452-08:00Great 404 page design from balsamiq<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiSViGXVH98DSWnPMxQXJHXQ6mydqd2k3c5iVBj3oV3tkMgGVQBDXuyW1Ubb4eBKynLR8aByCODMcbKRnkg6ZihqMJGrfKxYhBdoaG8AkAX2j1L-XgqoD_yFA9LVlgfaXjyYtVPah7eJkAG/s1600/Screen+Shot+2013-02-25+at+15.24.07.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="488" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiSViGXVH98DSWnPMxQXJHXQ6mydqd2k3c5iVBj3oV3tkMgGVQBDXuyW1Ubb4eBKynLR8aByCODMcbKRnkg6ZihqMJGrfKxYhBdoaG8AkAX2j1L-XgqoD_yFA9LVlgfaXjyYtVPah7eJkAG/s640/Screen+Shot+2013-02-25+at+15.24.07.png" width="640" /></a></div>
<br />Sam Josephhttp://www.blogger.com/profile/10788506730233381803noreply@blogger.com0tag:blogger.com,1999:blog-9148683707845127605.post-90255700068203053322013-02-20T02:52:00.000-08:002013-02-20T02:59:18.625-08:002013 Mac Book Pro Reset button is pressed a bit too easily ...<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhrsB_O22faTBgOAedtqNnRgTyLC9Lh2ipPJqabO67NUBJ6Ci3tB79gLlntx_fzBQPQlO8omn1UrR26tgJM6TPbavXIgJm2OWIuuj9sOBJHeBjnLAnBo5reRj9sgO6iZQ4-RwqfqwRzf4CY/s1600/Screen+Shot+2013-02-20+at+10.47.19.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhrsB_O22faTBgOAedtqNnRgTyLC9Lh2ipPJqabO67NUBJ6Ci3tB79gLlntx_fzBQPQlO8omn1UrR26tgJM6TPbavXIgJm2OWIuuj9sOBJHeBjnLAnBo5reRj9sgO6iZQ4-RwqfqwRzf4CY/s200/Screen+Shot+2013-02-20+at+10.47.19.png" width="162" /></a></div>
So in what may hopefully be an ongoing series of design bloopers here's something from my 2013 Mac Book Pro. The reset button was previously completely separate from the keyboard, but has now been placed just above the delete key, so that I accidentally hit it a few times a week. <br />
<br />
It does have a confirmation dialog to prevent me immediately shutting down my computer when I'm typing, but it seems like a bit of a hack. I guess there must be some strong technical reason for having changed this around, but I'm not seeing much in the way of improvement in terms of user experience here ...<br />
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiSMO7adsvzWbXdvla344vFH8U8ypE2JZIY71b-7S867381go3-Iot18qEGhQZp3i6uDJFUTAH3aJK3QhsGK9su-txr50C6RPzICw4xAQ4hq52io3dPqRnl_bEfh4RrqpDEKhBCQmN_BEEr/s1600/Screen+Shot+2013-02-20+at+10.50.04.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="113" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiSMO7adsvzWbXdvla344vFH8U8ypE2JZIY71b-7S867381go3-Iot18qEGhQZp3i6uDJFUTAH3aJK3QhsGK9su-txr50C6RPzICw4xAQ4hq52io3dPqRnl_bEfh4RrqpDEKhBCQmN_BEEr/s320/Screen+Shot+2013-02-20+at+10.50.04.png" width="320" /></a></div>
<div>
<br />
This seems to fit well with some of <a href="http://www.nngroup.com/articles/ten-usability-heuristics/">Nielsen's usability heuristics</a>. It does well in terms of "Help users recognize, diagnose, and recover from errors", but perhaps not so well in terms of "Error Prevention"<br />
<br />
<br />
<blockquote class="tr_bq">
<b>Error prevention</b><br />Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.</blockquote>
They've got the confirmation action, but better to design to avoid error occurring in the first place, and <a href="http://en.wikipedia.org/wiki/The_Humane_Interface">Raskin</a> hates confirmation dialogs ... It's particularly odd since the previous mac book pros were designed such that this was never an issue - the reset button was out of reach in a silvered portion of the body of the MBP in the top right, and was set into the frame so that it was difficult to press accidentally ... <br />
</div>
Sam Josephhttp://www.blogger.com/profile/10788506730233381803noreply@blogger.com0tag:blogger.com,1999:blog-9148683707845127605.post-9898932937047330272013-02-19T08:45:00.002-08:002013-02-19T08:45:24.637-08:00How NOT to do email notifications<div class="separator" style="clear: both; text-align: left;">
So in an effort to collect user interface (UI) bloopers, here's a pet peeve of mine. Google Plus notifications. This is how they look in the subject line view of Thunderbird. Look at this awful replication of information. The From field is telling me who it is from and that it's via Google+. The last thing I want is the same information in the subject line.</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgXTw7k9M9RQqrVvPkjZGYuP0aUsZg5nq-FP6DSD7XsGYk_XBf5nGjAch4-trpjsB7mlW7_YOkDWw8-VQWyKavEvyRnwGIRlK8HeQqNI4aJAMfgzwi4sy9D3ytdRMh246k6GiGqsTz_9RFJ/s1600/Screen+Shot+2013-02-19+at+16.27.13.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="249" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgXTw7k9M9RQqrVvPkjZGYuP0aUsZg5nq-FP6DSD7XsGYk_XBf5nGjAch4-trpjsB7mlW7_YOkDWw8-VQWyKavEvyRnwGIRlK8HeQqNI4aJAMfgzwi4sy9D3ytdRMh246k6GiGqsTz_9RFJ/s640/Screen+Shot+2013-02-19+at+16.27.13.png" width="640" /></a></div>
I don't think anyone could argue (in the context of this interface) that the subject line should be anything other than something about the contents of the post - the first sentence perhaps? The title of an associated URL that was being shared. Anything to let me know something more about what I might discover when I click through.<br />
<br />
Perhaps the problem is that the Google plus notification team hasn't optimized for Thunderbird? Fair enough - I can see Thunderbird users as a bit of a niche group, perhaps they are thinking in terms of the gmail interface:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3f_IZn7vei0e-VDd171f6s3nk-FWVb8yd5qlXGvmnWVGIYyyvFSp9YlXXJ1PgTTjgwoyAmsPzye4KrbC5MeuuNW-6wJ-5jEk662-9b4ukUwrVeSIoZKf0KVEDkScALF9yj2i767fVS4I1/s1600/Screen+Shot+2013-02-19+at+16.31.42.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="232" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3f_IZn7vei0e-VDd171f6s3nk-FWVb8yd5qlXGvmnWVGIYyyvFSp9YlXXJ1PgTTjgwoyAmsPzye4KrbC5MeuuNW-6wJ-5jEk662-9b4ukUwrVeSIoZKf0KVEDkScALF9yj2i767fVS4I1/s640/Screen+Shot+2013-02-19+at+16.31.42.png" width="640" /></a></div>
<br />
It would appear not - exactly the same problem in gmail. Although to be fair when I haven't searched for Google+ the faded out text on the right hand side does have a small amount of text from the actual post. Although if we are on search in gmail, why isn't there a sensible way for me to get back to the full inbox other than clicking inbox after a search - if I delete the search term in the box and press search shouldn't I go back to my full inbox?<br />
<br />
Anyway maybe these Google plus notifications are designed for viewing on a mobile device? This is what it looks like on my Nexus:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjua0qIJyI7kz2MQWRFvKXOmhDhRQZ9IhSSfXRReACNDd-WQcIlUegsT7BUMKTCpR2YW1YczEMbG9IHKHaJdZehL0a34fJkrD_RtBGwUjuOlOa3hQN2JIBB_WYV1iBuQ-WLZmmt43Sh02QG/s1600/Screen+Shot+2013-02-19+at+16.43.01.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="160" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjua0qIJyI7kz2MQWRFvKXOmhDhRQZ9IhSSfXRReACNDd-WQcIlUegsT7BUMKTCpR2YW1YczEMbG9IHKHaJdZehL0a34fJkrD_RtBGwUjuOlOa3hQN2JIBB_WYV1iBuQ-WLZmmt43Sh02QG/s400/Screen+Shot+2013-02-19+at+16.43.01.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Come on Google - this is really simple stuff no? I know I couldn't pass your complex engineering exam, but when are you going to hire me to sort out these UI issues for you ...</div>
<br />Sam Josephhttp://www.blogger.com/profile/10788506730233381803noreply@blogger.com0tag:blogger.com,1999:blog-9148683707845127605.post-39004199619226061782013-02-14T03:05:00.002-08:002013-02-14T03:07:25.947-08:00Harrow Local Support StakeHolders<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTVfut4R58FI7hupK-4akIToARn4KfSjRGUVKwdo6ull_gLeBU1fxGJqGJPVXy-EHvNq4ZMfGTeg_0JTDWR5dD3z6fdAqYmeXuqx4vJSXJ6b0EX9xxDIDg9x4aFuszNgdJfELi7ltnpbGP/s1600/5201843170_e705341776.jpeg" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="156" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTVfut4R58FI7hupK-4akIToARn4KfSjRGUVKwdo6ull_gLeBU1fxGJqGJPVXy-EHvNq4ZMfGTeg_0JTDWR5dD3z6fdAqYmeXuqx4vJSXJ6b0EX9xxDIDg9x4aFuszNgdJfELi7ltnpbGP/s320/5201843170_e705341776.jpeg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Image by <a href="http://www.flickr.com/photos/jurgenappelo/5201843170/">Jurgen Appelo</a> thru <a href="http://creativecommons.org/licenses/by/2.0/deed.en_GB">CC BY 2.0</a></td></tr>
</tbody></table>
One of my continuing posts related to the Systems Analysis / Software Engineering class I'm teaching this semester. Here's a list of stakeholders for a "Harrow Local Support" website/directory concept I've been working on with Voluntary Action Harrow (VAH) an umbrella group for local charities and non-profits.<br />
<ul>
</ul>
<div>
Harrow residents</div>
<div>
<ul>
<li>Looking for support</li>
<li>Looking for volunteering opportunities</li>
</ul>
<div>
Harrow Charities</div>
</div>
<div>
<ul>
<li>Trustees - who oversee the actions of a charities</li>
<li>Administrators - who run the charity</li>
<li>Staff - who work for the charity</li>
<li>Volunteers - who devote their time/resources for free</li>
</ul>
<div>
Harrow Non-profits (groups without charitable status, and also ???? unclear if there is a legal definition)</div>
<div>
<ul>
<li>Administrators</li>
<li>Staff</li>
<li>Volunteers</li>
</ul>
</div>
<div>
Voluntary Action Harrow (VAH)<br />
<ul>
<li>Administrator</li>
<li>Staff</li>
<li>Volunteers</li>
</ul>
</div>
<div>
Other organizations</div>
<div>
<ul>
<li>Harrow Businesses</li>
<li>Harrow Council</li>
</ul>
<div>
Stakeholders sounds a bit management faddish, but I guess it's a reasonable term that distinguishes itself from alternatives such as "people" and "users", which can lead one down the path of imagining a queue of people lining up to use whatever system one happens to create. Realizing that everyone is busy playing parts and roles in all sorts of other groups is important, and stakeholders captures the idea that individuals (and perhaps organizations) will have varying degrees of "stake" in whatever project you are developing.</div>
</div>
</div>
Sam Josephhttp://www.blogger.com/profile/10788506730233381803noreply@blogger.com0