Dropbox Drops Public - effect on Online Students

So it turns out the Dropbox removed the Public folder for new users a few months back.  This actually threw me and a lot of my students - for several months I have been continuing to tell my students to use their public dropbox folders, and it has only just come to light that they didn't have one unless they created it explicitly.   All good now, but it's a great example of poor visibility.  Not dropbox's fault as such.  Just something that's a problem with lots of online interactions - you can't see what the other person is looking at and you tend to assume that they see what you see.  If you've got a Public folder you assume others have.  You can say, well the students should tell you, but students are often overwhelmed and tend to assume, erroneously, that the instructor knows what they are talking about ... :-)

The Dropbox Public folder can still be created by any user who wants one and they are really really useful for quick testing of webpages, sharing images and so forth.  So useful in fact I wish I could adjust the Ctrl-Shift-4 screen capture on OSX so that any screen capture would be immediately moved to my public dropbox folder and then the public url for it be added to my clipboard, so that I could go Ctrl-Shift-4 and then Ctrl-V into an IM like Skype text chat to immediately share what I am seeing with whoever I happen to be chatting to at the time.

What's interesting to me is how long it's taken to work out that this was a problem.  The cut and paste screen shots feature that I mention above has been implemented in Linux by one of my past students, although it worked specifically with his own server and not with Dropbox.  It's a potentially great solution to the problem of poor visibility when working with others online.  Screen share via Skype or Google Groups is a great way to debug these issues, but it takes time to setup and the tendency is not to go there unless an obviously serious problem comes up.

I've noticed myself starting to prefer Skype text chat recently over email recently.  Some people have laughed about that, "yeah right, because starting up an email client is so hard", however I think there is something important there.  It's like being in the same room with someone and just being able to turn around and talk to them, versus needing to pick up the phone to call them.  To send an email I have to go to my email client (I use Thunderbird and a few Cmd-Shift clicks on OSX will take me there), get a new message window open, start typing the person's email address, and then a subject line and then get my cursor in the main window.  It's not huge, and perhaps it would be simpler in some webmail set ups, but it's all little cruft actions that get in between my thought and getting it across to someone.

For IM chat (in my case, usually Skype text chat) I just Cmd-Shift to that window and start typing.  I may need to click on a different chat room, but it's pretty damn fast.  Of course the dynamics of both email and IM change if you are actually responding to someone, but I really do find myself feeling like their is a lower barrier to the IM chat, and by arranging the groups of people in the different Skype chat rooms appropriately there's much better visibility than email, although that depends on how people are using Skype.

Some might argue that the lower barrier is a bad thing, causing myself, colleagues and students to spend all their time blathering in Skype text chat.  Also, as Dave Patterson mentions in CS169.1X, there is the danger of communication addiction, where you are constantly seeking little dopamine squirts and find yourself checking for messages all the time.

However, this issue of Dropbox dropping the public folder only finally became clear to me as a result of chatting with a student in Skype text chat.  Maybe it would ultimately have come out in an email or a forum post, and maybe it distracted me from doing something else important, but having at least a partial window into what students are going through as they try to complete assignments seems like it might be the most crucial thing for an online instructor.  Or perhaps I'm just deluding myself in order to get another dopamine fix :-)

My Mobile Design & Programming Course Available Publicly

So Hawaiʻi Pacific University (HPU) has very kindly given me permission to take my Mobile Design and Programming class public.  The class is already under way and has 9 HPU students and 6 external students.  It's not exactly a MOOC (Massive Open Online Class), more of a SOOC (Small Open Online Class).  I'm quite happy with it that way for the moment.  I'm testing out some new features such as an assignment self-tracking system (see image below) that I developed using Google App Scripts and plugged into Google Sites, and I think it's good to start small with these things.

Also, I think the key thing about classes is being able to have meaningful interactions with the other class members, which is challenging in MOOCs.  I'm hoping that I can gradually scale this class up to more students while maintaining a tight knit feel.  In order to do that I think we have to start small and gradually work our way up, rather than starting with 40,000 students and then trying to work back from there.

Some things I am happy with are having migrated my class syllabus purely to HTML (goodbye word documents), shared all my materials under Creative Commons BY licensing and being explicit about learning objectives on a week by week basis.  I'm also continuing to enjoy having all my lecture videos on YouTube, Skype chatrooms and having all my students submit assignments on their own personal blogs. I think this last is the critical thing that may allow this more personal (POOC?) type of course to scale.

The problem with MOOCs is that they hope all the students will spend time posting internally into their forums, which creates a great cache of content inside the MOOC for the MOOC authors, but I feel the value proposition is not so good for the students.  By posting on their blogs students associate their assignments and opinions more strongly with themselves; they build up a portfolio of their work that can exist, and be discovered, outside the MOOC.  The student can still control their blog visibility, and they can always go back and revise any content they are not happy with.  Rather than a certificate at the end of the course, the students have a fully browsable blog documenting their creative process - I know which I'd prefer :-)

Brainstorming Waiting in Line ideas

So HPU has very kindly given me permission to take my Mobile Design & Programming class public. I haven't been promoting it as I think it's not ready for 100's of learners, but it's great opportunity to allow HPU and non-HPU students to mix, and also just to expose all of the class internals to broader inspection.  If you'd like to sign up do get in touch.

Anyhow, the key week 2 assignment is about Human Centered Design based with permission on one of Scott Klemmer's great assignments from his CS147 HCI Class at Stanford.  And to put my money where my mouth is, and to give my students an example of a complete submission, here's my own submission.  The assignment is to brainstorm ideas to help improve the experience of waiting in line, and choose two, quickly prototype them on paper, then test them out with real people waiting in a line.

First up, my brainstorming of 20 ideas for waiting in line activities

1. talking to jokebot - an app that chats with you and tries to make you laugh
2. reminders to take your vitamins - kudos to previous mobile design students who had this idea
3. browsing coupons for the supermarket
4. crosswords, sudoku -games
5. hyperlocal news - an app or map layer that shows you wants going within 500 yards
6. help you wait calmly - calming music?
7. connect with friends - I guess we have fb already
8. yoga, posture exercises
9. teeth - teeth cleaning exercise
10. Buddhist sutra of the day
11. Game exploring Jomon period lifestyle in neolithic Japan
12. App to show you what your current location looked like at different points in history
13. Imagining different points in your life - how does now compare to when you were 5? will likely be when you are 70?
14. listen to invigorating music to get you pumped up for the day
15. meditate on a spot within your field of vision
16. wonder what it means for a color to be a color - what actually is green?
17. start a conversation with the person next to you on the weather
18. ask the person next to you "what's the meaning of life"
19. flex your abs/pecs exercises
20. plan the remaining part of your day

So quite a few of the above just end up being things you could do while waiting and don't necessarily require any scaffolding whatsoever, although one could imagine mobile apps for many of them, and in many cases they already exist.  Makes me think that one could have a "waiting in line" app that gave you a random suggestion of things to do. Of course the critical thing with brainstorming is just get ideas out, however goofy they are and worry about tidying them up afterwards :-) 

Anyhow, my two favorites are 

1. hyperlocal news - an app or map layer that shows you wants going within 500 yards
2. talking to jokebot - an app that chats with you and tries to make you laugh

Which are actually things I've kind of been working on in the background of my life on and off for a while now.  That's perhaps a bit sneaky, but hey I think you need to link assignments up to things you are interested in :-)

So some more explanation on these two ideas, is that I'm fascinated by humor, and the idea that computers might be able to simulate it.  So much so that I started the Funniest Computer Ever contest. Also the idea of hyperlocal geotagged news is something I've worked on in the past.  I'd love to get news chunks displayed on a local map, showing me just where thinks happening close to home are going on.

So here's a couple of really fast paper prototypes of the concepts, using sharpies on a set of 3x5 cards with a tiny single ring binder to hold them together:

1. HyperLocal News



2. JokeBot - jokes from the internet joke database



And here's videos of the user testing:

So I've diverged slightly from the assignment here in that rather than photos I've gone with videos.  I wussed out slightly in that I didn't go up to complete random strangers, but I did recruit people who I had never previously met at an Interactive Narrative event in London.

Immediate feedback was need for "back buttons" on the hyperlocal news app.  Using my trusty Sharpie I quickly added back buttons for the HyperLocal News app.

Overall I was probably talking to much, although I wanted to simulate audio for the jokes, which I would imagine being on the actual device.  Although that's something I need to keep working on, i.e. as Scott Klemmer points out, a bit of silence is golden.  I'm terrible at it - ideally I'd be behind a glass screen unable to tell people what it is they are seeing.

It was interesting that everyone clicked on the dirty jokes option and then animals.  One person said they were uninstalling the joke app.  Another person said the dirty joke button would be selected because it was in the middle.

Another observer mentioned wanting to be able to skim all the news stories without having to jump back and forward, so to enumerate my insights from this round of testing:

1. Navigation critical, must have back buttons or other obvious means of navigating data
2. Could we provide swipe based navigation to completely avoid jumps?
3. I'm assuming jokes really need audio ..., but do they?  I didn't allow myself to find out
4. People like dirty jokes ...?
5. Everyone asked about the practicality of the hyperlocal news app.  One person mentioned the Open Data Institute as a resource, and there is an RSS news feed for my local area, but the RSS elements are not geotagged

Also I wonder if rather than saying, "imagine this is a touch screen phone", it would be better to put photos of the cards on real phone that would allow simple swiping ...

Criticism's of Agile Development by Dennis et al

In their book "Systems Analysis and Design with UML" 4th Edition, Dennis et al discuss Agile Development, and mention the following criticisms:

1. Today's business environment, where much of the actual information systems development is offshored, outsourced, and/or subcontracted does not fit well with agile development methodologies requiring co-location of the development team
2. If agile development is not carefully managed, and by definition it is not, the development process can devolve into a prototyping approach that essentially becomes a “programmers gone wild” environment where programmers attempt to hack together solutions
3. A lack of actual documentation created during the development of the software, raises issues regarding the auditability of the systems being created. Without sufficient documentation, neither the system nor the systems-development process can be assured.
4. It is unclear whether agile approaches can deliver large mission-critical systems

My immediate reaction to point 1 is "Google Hangouts", "Skype Screenshare" etc.  It seems to me that one can work almost side by side with other people around the world now.  Regarding point 2, I think this is a danger for all software projects.  I don't know that I agree that agile methods are not carefully managed.  It would seem to me that careful management is an orthogonal issue to agile development.  You can have a large waterfall development process that is not carefully managed, and ends up generating reams of useless documentation, and you can have a carefully managed agile project that keeps the developers talking to the customers, and any documentation generated carefully in step with needs and software alike.  

Regarding point 3 agile methodologies emphasize working software over documentation, but that doesn't mean there should be no documentation.  Dennis et al appear to be arguing that some critical mass of documentation is required in order to perform an audit, and assure the system and systems-development process. I don't see why this should be the case.  Surely an audit can be performed on a working system as well as on documentation.  Futhermore, isn't the bigger danger that one has lots of documentation that is unrelated the actual system or indeed the actual needs of the users.  In this case what does the documentation help us assure in terms of the system or systems development process.

Regarding point 4 this seems like a open question.  One might well argue that large payroll systems, or systems like the mars rover must be developed much more carefully than a social media app for a mobile device, but in all cases if the resulting system fails to meet user needs then there are no winners. Personally. I am skeptical that the big design up front waterfall model necessarily produces good results on large mission-critical systems.

To be fair, Dennis et al go on to say:

Even with these criticisms, given the potential for agile approaches to address the application backlog and to provide timely solutions to many business problems, agile approaches should be considered in some circumstances.

So all credit to them for including the agile ideas in their systems analysis textbook.

Tom Gammarino's Free Writing Challenge

So as part of my game programming course, guest imagineer Tom Gammarino challenged my students to a 10 minute free writing challenge, where they write on any topic for 10 minutes without stopping, and I thought it important to take the challenge myself.  My ten minutes of writing without stopping became 15mins and I include it below.  I had planned to do something more narrative based, but it became more about some thoughts about games I am wondering about being able to create.  Either way it was fun to force writing like this - I wonder if I could do it for longer - I always remember that author who thought he was going to die (Anthony Burgess?) who tried to write all his remaining novels in a year and then didn't die and just carried on writing at some ridiculously prolific rate - anyway, want to try this again and force myself into some more narrative thing where I am describing something that is happening to some set of characters ...

There was a young man from nantucket, I like limericks and IO'd like to implement a computer system to generate them ,but I just don't seem to get time for these thethings during the normal course of he semester, and I want to create a computer game interactive narrative that explores the development of Japan during the Jomon period, which is a really interesting period since they seemed to develop advanced pottery before some other parts of the world ,who were still in the new stone age or something.  of course the astounding thing is the suggestion that we can really know anything about what people's lives were like during that time.  I'm reading these japanese history manga and the early lives seem pretty idyllic in terms of hunting and gathering and building small villages and starting to use agriculture, but pretty soon we get into the later yayoi and kofun periods and before long there is warfare and invading korean and things like that - seems like agriculture supported a huge increase in population and that ultimately led to people who had leisure time for warfare and inequality and things like that.  Of course before the start of organized hostilities there would have been lots of unpleasant situations like not managed to get enough to eat, and your kids dying from starvation, and getting injured going hunting and starving to death.  It would just be so fascinating to actually go back in time and watch some people living in one of these periods that we only really know anything about by archaeology and inference based on observing indigenous tribes - of course I guess the thing to do would be to become an anthropologist and go off and live with some of these pre-industrial tribes that still live in the amazon rain forest - I did take an anthropology course at edinburgh university one time and the instructor had lived in the rainforest with one group and talked about his experience of being a child, like they couldn't leave him along in the forest as he would get totally lost - he didn't have any independence and was totally reliant on the tribesmen to get him to places and to feed him, since he had none of the skills that allowed him to survive in the forest - of course it's the gradual introduction of technology and adoption of new techniques for getting food etc. that's just absolutely fascinating which is what we see in games like civilization and makes them real fun, although they do still seem to lack something of a narrative.  I ilke these japanese history comics because they give a bit more of a narrative sense, but it doesn't go quite far enough - they introduce some early historical figures like some queen who sent a messenger to mainlinad china in the 2nd century, but we don't really know hardly anything about her, and so they kind of sketch her out as a kind of intelligent young woman who was really good at predicting the weather, and who was raised up to be queen to avoid the men all fighting each other to be king, but we really don't know the politics of the events of the time, and I guess they are afraid to make them up since they are trying to be historically accurate books/comics things, but I think they could go further, but then I am also complaining that they don't really know how people lived although I haven't followed up on all the background sources that they might be using.  but there's a difference about developing a narrative.  There was a book, "clan of the cave bear" or something that explored that - it was like a story set in the stone age, and "American Gods" had some of that in it's little pre-history vignettes - I still think an interactive narrative in the Jomon would be cool, but I'd really like an underlying physics model so that game play was kind of open ended, e.g. you could make clay into pots and discover some early technologies, but in a more narrative oriented fashion than civilization, e.g. you can store more nuts and berries in your pots in order to make it through the winter, I mean get your little family to make it through the winter - I can imagine a really raw game where sometimes you'd have to make choices like leaving a sick child to starve in order to have your other healthy children survive - would that make for fun gameplay? I'm not sure - it's certainly a very dramatic issue and something that early humans likely had to face - there was a song I learnt in a language school in japan that was said to be dedicated to those children that had to be killed so their brothers and sisters might survive.  It seems so horrific in this modern age where we expect all of our offspring to make it to maturity - is there value from exploring these things?  perhaps in just making us appreciate how lucky we are now ...

Code Complete: High Quality Routines

So reading Steve McConnell's Code Complete chapter 7 on writing "High Quality Routines" I am struck that there is lots and lots of great advice in there about writing methods.  Particular things such as clear naming conventions, relatively short routine length, short sets of input parameters etc. are all excellent.  However the problem that I have with the chapter overall is that it feels like it is easy to read it and then not put much of the advice into practice.  While the chapter has lots of great example code snippets, I feel like it is easy to ignore the great advice, given that it's not being encountered as part of a larger project.

Maybe this is just me.  I'm a huge fan of working on things in a larger context, but I know that that's not everybody's cup of tea.  It has been argued that some people like to read the abstract theory associated with things before encountering examples in context.  If you're one of these people please do get in touch and let me know.  I'd like to learn more about your thought processes.

Anyhow, just as an example of some mixed routine practices from things I was programming today, I created one short routine
and grabbed another from StackOverflow:

http://stackoverflow.com/questions/8240637/javascript-convert-numbers-to-letters-beyond-the-26-character-alphabet

Now I've got a really awful vanilla name for the routine I created (processForm) and all sorts of hacked together pieces of things, but that's partly a reflection of the "tracer bullet" methodology I was working on here, which was to get something that would take a form submission from a Google App Script generated html page, and enter the resulting data into a spreadsheet as described in this StackOverflow post:

http://stackoverflow.com/questions/13086880/can-i-mix-jqueryui-and-google-app-script-form-submissions

At least I can be pleased that both routines are relatively short and have a very small number of input parameters.  I think that both could be improved with refactoring.  I'll present that in a future blog post, but right now I know I and my students are going to be using this code every week to support assignment submissions, and getting a feel for getting this working in Google App Scripts was the top priority that trumped every other consideration.  Not the best programming practice perhaps, but let's see how I can iterate and improve this over the next few weeks.

Google App Scripts really rocks, but I'm still trying to work out how to version and share all this stuff effectively so we can see the software engineering and systems analysis issues in the context of this larger project, as described in this other SO post:

http://stackoverflow.com/questions/12712593/should-google-app-scripts-be-stored-in-version-control-like-github

Automated Text Chat Assistants for Online Classes

I'm proposing this research project at my University in the hope of getting some course releases to spend more time on it. The proposed project concerns analyzing the text and email logs from a number of online computer science and multimedia communication classes in order to identify common students questions and concerns.  This data will be used to improve course design and automate the generation of frequently asked questions (FAQs) web pages.  Longer-term goals include the developing an instant messenger (IM) chatbot that can interact with students online when instructors are not available.

Background:

Researchers have studied the effects of making synchronous chat available in online classes such as Spencer & Hiltz (2003) who showed that students found classes with synchronous chat significantly more ‘rewarding’ and less ‘complex’ than classes with only asynchronous communication. This is not to say that asynchronous interaction such as that facilitated by blogs, emails, bulletin boards, wikis and so forth it not valuable.  In fact according to Johnson (2006) both synchronous and asynchronous forms of online discussion have advantages and there is evidence that both contribute to student cognitive and affective outcomes.

While the benefits of rich interactive tools for online learning are not in question, there have been relatively few assessments (although see Tomei, 2006) of the time taken by the online instructor to manage these systems.  The principle investigator of this project argues that much instructor time could be saved by the automated generation of class frequently asked question (FAQ) pages, as well as by providing multiple means to access this information, possibly including chatbot technology to interact with students over instant messenger (IM) systems such as Skype.

There have been a number of attempts to integrate chatbot technology into the online classroom over the years.  For example Heller et al. (2005) demonstrated that famous person chatbots (e.g. FreudBot) are promising as a teaching and learning tool in online education; while Mikic et al. (2009) developed the CHARLIE chatbot which can maintain a general conversation with students, showing them contents of courses, and asking them questions about learning material.  Even without a working chatbot there are studies (Kerly et al., 2006) using Wizard-of-Oz experiments that indicate this technology, if successful, could have widespread application in schools, universities and other training scenarios.

There are some positive results at the K-12 level (Kerly et al., 2007) however other authors (e.g. Knill et al., 2004) have cautioned that:

The teacher is the backbone in the teaching process. Technology like computer algebra systems, multimedia presentations or ‘chatbots’ can serve as amplifiers but not replace a good guide.

Despite these concerns recent researchers have experimented with humor enabled chatbots (Augello, 2008) and chatbots that support second language learning (De Gasparis & Florio, 2012; Jiyou, 2009; Sha, 2009); while yet others have demonstrated chatbot effectiveness in supporting network management training (Leonhardt et al., 2007) and security training (Kowalski et al., 2009). Furthermore in a detailed study Alencar & Netto (2011) developed an online education portal chatbot that successfully answered 69% of the students’ general questions about distance education.

In summary it seems that there is a great potential for the use of chatbot technology in the online classroom, but that one should be careful not to try and replace the human instructor entirely.  The proposed project is focused on providing automated online teaching assistants that will help the human online instructor manage their online classes. The principle investigators experience indicates that synchronous chat helps with student engagement and retention although as Wilging & Johnson (2004) point out students’ reasons for dropping out of an online program are varied and unique to each individual.  It is the principle investigators hope that well designed automated FAQs and IM chatbots can server to amplify the retention-enhancing effect of synchronous chat in online classes.

Clearly a cautious approach is required to develop systems that can successfully support online learning success.  This project plans to analyze the logs from several online classes using the ethnographic approach demonstrated by authors such Zembylas & Vrasidas (2007).

Work Plan:

Jan/Feb 2013: Analysis of CSCI 4702, 4705 and MULT 4702 chat logs
Feb/Mar 2013: Generation of 1^st version of FAQs
Mar/Apr 2013: Automation of FAQ generation
Apr/May 2013: Link existing chatbot to FAQs to support usability testing over summer 2013

Work and Funding so far:

This project has not previously received any funding, but the principal investigator conducted an exploratory analysis of the chat logs during summer 2012.  A basic chatbot that answers simple questions has been created and deployed and further analysis and development is required to make something that can be of use in an online course.

No funding has been received or applied for from other sources.  The project is ongoing and will continue as long as the principal investigator continues to teach online classes.  However in the first instance a single course release is being requested to help work through the large volume of text chat that needs analyzing

Budget Justification:

The budget is for a single course release to provide the principal investigator with time to conduct the chat-log analysis and do code development based on the results.

Access:

No specific facilities are required.  The only requirement is IRB approval which is pending from the HPU CHS.

Dissemination:

Dissemination includes presenting the work at professional conferences such as the World Congress on Education or the International Conference on Advanced Learning Technologies, and ultimately in journals such as Knowledge-Based Systems and Distance Education.  The work would also be disseminated as an open source package and be promoted through the “Funniest Computer Ever” annual computer comedy contest. 

In the long term it would be good to see many educators making use of the techniques and tools developed from this project, and the plan is to disseminate the work as widely as possible through both academic, i.e. peer-reviewed publications and technical circles, i.e. an open source toolkit for online educators.

Deliverables:

Two conference paper submissions and working FAQ and chatbot prototypes by end of Summer 2013, for deployment in Fall 2013 classes.

Bibliography

• Alencar, M. & Netto, J. (2011). Developing a 3D Conversation Agent Talking About Online Courses. In T. Bastiaens & M. Ebner (Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2011 (pp. 1713-1719). Chesapeake, VA: AACE
• Augello, A. (2008)Humorist Bot: Bringing Computational Humour in a Chat-Bot System. International Conference on Complex, Intelligent and Software Intensive Systems, 2008. CISIS 2008.
• De Gasperis G.  & Florio N. (2012) Learning to Read/Type a Second Language in a Chatbot Enhanced Environment. International Workshop on Evidence Based Technology Enhanced Learning. Advances in Intelligent and Soft Computing, 2012, Volume 152/2012, 47-56
• Heller, B., Proctor, M., Mah, D., Jewell, L. & Cheung, B. (2005). Freudbot: An Investigation of Chatbot Technology in Distance Education. In P. Kommers & G. Richards (Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2005 (pp. 3913-3918). Chesapeake, VA: AACE.
• Jiyou J. (2009) CSIEC: A computer assisted English learning chatbot based on textual knowledge and reasoning. Knowledge-Based Systems. Volume 22, Issue 4, May 2009, Pages 249–255
• Kerly, A., Ellis, R. & Bull, S. (2007). CALMsystem: A Conversational Agent for Learner Modelling, in R. Ellis, T. Allen & M. Petridis (eds), Applications and Innovations in Intelligent Systems XV – Proceedings of AI-2007, 27^th SGAI International Conference on Innovative Techniques and Applications of Artificial Intelligence, Springer Verlag 89-102.
• Knill, O., Carlsson, J., Chi, A., and Lezama, M. (2004). An artificial intelligence experiment in college math education. Preprint available at http://www.math.harvard.edu/∼knill/preprints/sofia.pdf.
• Kowalski S., Pavlovska K. & Goldstein M. (2009) Two Case Studies for Using Chatbots for Security Training. World Conference on Information Security Education 2009.
• Leonhardt, M.D. Tarouco, L., Vicari, R.M.,Santos, E.R. & dos Santos da Silva, M. (2007) Using Chatbots for Network Management Training through Problem-based Oriented Education. Seventh IEEE International Conference on Advanced Learning Technologies, 2007. ICALT 2007.
• Shaa G. (2009) AI-based chatterbots and spoken English teaching: a critical analysis. Computer Assisted Language Learning, Volume 22, Issue 3.
• Spencer, D. H., & Hiltz S. R.  (2003) A field study of use of synchronous chat in online courses. Paper presented at the 36th Hawaii International Conference on System Sciences, Big Island, HI, January. http://csdl2.computer.org/comp/proceedings/hicss/2003/1874/01/187410036.pdf
• Tomei, D.L. (2006). The Impact of Online Teaching on Faculty Load: Computing the Ideal Class Size for Online Courses. Journal of Technology and Teacher Education, 14(3), 531-541.
• Zembylas, M & Vrasidas C. (2007) Listening for Silence in Text-Based, Online Encounters. Distance Education; May 2007; 28, 1.