This week we decided to include additional monitoring features in the web app. For example, I will now be logging mouse dragging movement and keyboard strokes. I will also time how long it takes for the users to complete the tasks. And if the time in the experiment section is shorter and the quality of the answers is the same, that means it is better to filter based on constructiveness. This is will validate the usefulness of identifying constructive content.
I am still implementing the user study and hope to have it finished by the end of next week so we can start recruitment. We will need 48 students to complete the study. Thus, I have created an advertisement flyer and have started recruiting some eligible candidates. By the end of next week, I will also have run through the user study with a test subject (another member of the lab) to make sure everything is running smoothly.
This week I have been busy continuing my implementation of the user study in python and django. Here is some progress!
We have revised the design of our user study a little bit more. This week I have been brainstorming questions and problems that have to be completed at the end of each task to capture immediate learning. I will be removing the post-test at the end of the user study, since it does it help us to differentiate the learning that occurs at the end of every task.
This week it was my turn to present in the weekly CSI for Learning Lab meeting today. I presented on purpose and design of my user study and was received valuable input from others in the lab group on how to keep some of the variables constant to make sure they are not affecting the results.
This week I have been continuing to implement the web app. I have definitely learned that design is a very important aspect of the user study. It is important to make everything look nice and professional to not distract the user. Fixing CSS problems can also be very time consuming.
Through my Human-Computer Interaction (HCI) class, I have learned about intrinsic and extraneous cognitive load in learning and performing tasks. This is very relevant to my project and as a result I have started reading more papers about those topics. In terms of HCI, the purpose of my user study is to eliminate as much extraneous cognitive load as possible by removing and filtering content which is not conducive to learning.
During winter break, I started working on a web development project with two other students using python's django framework. I learned a lot more about using bootstrap as well. I will be switching over to django to implement the user study as well since it makes web programming a lot easier and cleaner.
This week, the entire CSI for Learning lab was assigned to write a 2 page report to practice scientific writing. Dr. Hsiao gave a presentation on structuring a project. We went over the some of the different parts of a project and a paper: Research Question, Methodology, Evaluation, Literature Review, and Communication. I wrote my two pages on the user study which I have been designing this semester.
This week we decided to redesign our user study to include a second control group.
Control0: In this group, all the answers to StackOverflow questions will be ordered randomly. This will help validate that the user really is knowledgeable enough to reasonably score answers based on usefulnesses. Without this group, it will be hard to determine the credibility of a user’s ratings since the user might be inclined to rate based on his or her perception or anticipation of what the score should be instead of the actual content.