The PSLC DataShop provides two main services to the learning science community:
- a central repository to secure and store research data
- a set of analysis and reporting tools
Researchers can rapidly access standard reports such as learning curves, as well as browse data using the interactive web application. To support other analyses, the DataShop can export data to a tab-delimited format that can then be used in statistical software and other analysis packages.
Read the “Guide to the Tutor Message format” —
HTML or PDF
more information
DataShop News
Wednesday, 9 July 2008
Patch Released — DataShop v3.0.22
Many bugs fixed but we have a few more to go. See the list of known issues.
Wednesday, 18 June 2008
Patch Released — DataShop v3.0.18
Fixed many of the high priority bugs in the Known Issues list. Be on the lookout for more patches to come this week and next until we have all High and Medium issues resolved.
Wednesday, 4 June 2008
Announcing DataShop v3.0 — Released and Ready to Use
We have a ton of new features to tell you about, so sit back, relax, and read on...
- Create and import your own KC models
- Auto-generation of generic KC models
- Latency Curves
- New version of LFA
- Student-Problem Rollup
- Contextual Help
- KC sets
- and much more...
Create your own KC models
From the new KC Models page (Dataset Info > KC Models), you can view existing KC models, export an existing model or template for creating a new KC model, or import a new model that you've created.
You might create a new KC model because an existing model is insufficient in some way—it may model some knowledge components too coarsely, producing learning curves that spike or dip, or it may be too fine-grained (too many knowledge components), producing curves that end after one or two opportunities. Or perhaps the model fails to model the domain sufficiently or with the right terminology. In any case, you may find value in creating a new KC model.
By importing the resulting KC model that you created back into DataShop, you can use DataShop tools to assess your new model. Most reports in DataShop support analysis by knowledge component model, while some currently support comparing values from two KC models simultaneously—see the predicted values on the error rate Learning Curve, for example.
Auto-generation of generic KC models
DataShop now creates two knowledge component models in addition to the model that was logged or imported when the dataset was created:
- Single-KC model: the same knowledge component is applied to every transaction in the dataset, producing a very general model
- Unique-step model: a unique knowledge component is applied to each unique step in the dataset, producing a very precise (likely too much so) model.
These generic models represent extremes (in terms of model complexity) within which your own KC models should fall. You can compare the AIC and BIC values of your own model to these generic models to gauge the "goodness of fit" of your model.
Latency Curves
The Learning Curves line graph can now be viewed by two new time-based measures:
- Assistance Time: The elapsed time of a step; the duration of time for which students are seeking assistance. (If there is no eventual correct attempt, the assistance time is the total length of time spent on the step.)
- Correct Step Time: The duration of time for which students are "silent", with respect to their interaction with the tutor, before they complete the step correctly. This is often called "reaction time" (on correct trials) in the psychology literature. If the first attempt is an error (incorrect attempt or hint request), the observation is dropped.
New version of LFA
We've updated the Learning Factors Analysis (LFA) algorithm from version 1.0 to 2.0, which includes two variants of LFA.
DataShop now runs the Additive Factor Model (AFM) variant of LFA when a dataset has 1 or 0 knowledge components associated with each transaction, and the Conjunctive Factor Model (CFM) variant of LFA when a dataset has transactions with more than one knowledge component associated with them. Both variants generate the same models, but AFM runs more quickly. For more information about these LFA algorithms, see our LFA documentation on "Predicted Error Rate".
The major changes in LFA 2.0 are:
- Faster parameter estimation (half of the time) and lower memory requirements (half of the memory) in the AFM version of LFA
- No model intercept—the existing model intercept was generating more than one model parameter per data point, increasing the model complexity without improving its fit to the data.
- The knowledge component slopes are constrained to have zero or positive values—negative KC slopes were modeling that the student did worse than not learn, and so were describing a better fit than would be reasonable. KC models for which students appear to be getting worse are now penalized (in AIC and BIC) more heavily due to the knowledge component slope minimum value of zero.
Student-Problem Rollup
In addition to the transaction and student-step roll-up exports, you can now roll up the data by student and problem—click Export > By Student-Problem. You can get problem start time, end time, latency, total number of hints/incorrects/corrects, total number of steps, etc. See the documentation for more details.
Contextual Help
Many users have wondered where to get help about DataShop's tools and algorithms. In response, we have added a nifty green button in the top left-hand corner of DataShop, right above the navigation boxes, where you can get contextual help on every tool and report. From there you can drill down deeper into our documentation.
KC sets
Tired of selecting the KCs you care about every single time you go back into DataShop? Well, the ability to save you favorite KCs into a set and share them with your fellow researchers is here! Click the wrench icon in the "Knowledge Components" navigation box to get started.
and much more...
So what else? A bunch of little things we hope you find useful:
- Moved the Performance Profiler tab so you know you should start there for exploratory analysis.
- Added the ability to page through the transactions in the export preview table.
- Undo on the Dataset Info page.
- The ability to minimize and maximize the navigation boxes on the left hand side.
- Displaying user and date on Papers and Files page.
- "Save as New" button on the Sample Selector dialog.
Still reading?
Wow, I'm impressed that we held your attention all the way to the bottom of this announcement. Please let us know if you like the new features. Also, please please please let us know what new features you'd like to see.