Saturday, January 28, 2017

Geoffrey Cohen's talk at Rice on Inclusive Teaching

Inclusive Teaching

I went to an "Inclusive Teaching" workshop by Geoffrey Cohen, who works with Carol Dweck at Stanford. The workshop was sponsored by Rice's Center for Teaching Excellence and was well attended by Rice faculty and staff. If you don't know Carol Dweck, she pioneered a branch of research on the effects of mindset on performance in a wide variety of settings, concentrating on academic achievement. In this model mindsets fall into two camps. A 'fixed mindset' is a belief that a particular trait, like intelligence for instance, is fixed at birth and basically cannot be changed, versus a 'growth mindset' which is a belief that a particular trait is malleable and improves with practice and effort. There are many, many different experiments that show that regardless of initial measured ability, a growth mindset is associated with higher performance academically over time, and this appears to be due to increased tenacity in the face of challenge, because failure is not seen as a measure of ability. Furthermore, particular interventions can shift a person's mindset and shifting that mindset results in increased performance. When these interventions work, the results are significant and can be long lasting, on the order of years.

Given the potential, I have been interested in how we might incorporate growth-mindset inducing features into OpenStax products, and whenever someone with good ideas and research is around I try to learn what I can from them. These are my notes from this talk.

Social belonging / Stereotype threat / White men can't jump

The talk concentrated on social belonging. You may have seen some of the research on what is called 'stereotype threat'. It seems counterintuitive, but it appears that if you think that people believe your group isn't good at something, and your performance could confirm that negative stereotype, your performance suffers. That is a very causal way of explaining it, and, of course, these are really just correlations, but now I will just tell you some of the weird and wooly experiments that have been done. All of these divide subjects as evenly as possible into two groups, one of which gets the 'treatment' (in this case a negative treatment) and the other of which doesn't, and then average scores are compared.
Things that decrease performance:
  • If you ask people to list their gender before taking a math test, female scores drop significantly.
  • If you ask people to list their race before taking an academic test, black student scores drop significantly. (There is such a thing as 'stereotype lift' also. White scores increase a little if asked to list their race, but the increase is much less than the decrease for groups where a negative stereotype is part of the culture).
  • If you tell people a test is a measure of intelligence, certain minorities and females do worse than giving the same test and characterizing it differently (skills …)
  • If a black researcher asks white men to jump as high as they can, they jump less high than if they are asked by a white researcher.

Digression: Unconscious bias in hiring

Cohen went in to a significant digression about experiments that show unconscious bias in hiring. I think this was mainly to give examples of how interventions can fix things that are unconscious and hard to just 'goodwill' away.

Research demonstrating bias

Specifically, there are a set of experiments that show that when comparing two candidates, people adjust their expectations to favor candidates that fit their stereotypes. For example, when presenting two candidates for a police promotion, one of which is male and one of which is female, and giving these candidates either more 'on-the-job' experience or more 'book-learning' experience, if you first show the candidates and then ask which is more important 'on-the-job' or 'book-learning', the hiring manager picks whichever criteria the male has.

Techniques that can decrease bias

But just like with mindset, there are interventions that can eliminate or improve these biases.
  • If you ask people to come up with the criteria for the best candidate before they see the candidates, they stick with those criteria and, in the case of the police promotion will (on average) pick a female candidate matching the stated criteria, over a male candidate that does not.
  • When people hire a group into a position, for instance hiring three managers at once, or three developers etc. - they are more likely to select a diverse group, than if they hire three people in successive rounds.

Social Belonging interventions that increase student performance

Then he came back to listing a set of 'interventions' that have been shown to have positive effects for females in male dominated fields, minorities in white dominated achievement areas, first generations college students, etc. These particular interventions did not show positive or negative effects for other groups, but studies that measure attitudes first do show benefits for all students coming in with particular mindsets and attitudes.
  • Having students read about 'real' students who felt they were not smart enough, or did not belong, but then found that they did. Or attend a panel of students discussing these feelings, especially if the audience identifies with the students (gender, race, economics, etc).  
  • Having students choose three sentences from among a long list that are important to them and then write a paragraph about each. Cohen called this 'value affirmation'. The values listed have a wide variety of things, and include non-academic values like 'sense of humor', 'relationship with family' (This intervention reduced F's in a course by 50%, from 20% to 9%).
  • For K-12 students, having a teacher write 'I am giving you these comments because I have high standards and I know that you can meet them.' to accompany corrections and comments on an assignment. Teachers pre-wrote these and research assistants attached them to student work. Teachers and researchers were blind to who got these and who didn't.
  • For K-12 students, having a teacher initiate an exercise where students write the end of this sentence 'I wish that my teacher knew that …'

This summary from Carol Dweck's website, Academic Tenacity: Mindsets and Skills that Promote Long-Term Learning, has more about a lot of the research that Cohen described.

Wednesday, December 7, 2016

Edit (Math) or Bust - Sprint Nov. 16 - 19 (Part 1)

Thanks to the Shuttleworth Foundation, where I had a fellowship from 2011 to 2014, OpenStax (my current employer), and all of the participants (listed below), I was able to host a sprint at the OpenStax offices Nov. 16th - 19th to investigate easier ways to edit and convert mathematics within open textbooks, as well as to make it easier to adapt and customize OpenStax college textbooks.

Participants sitting in a u-shape with laptops, and large screen showing demo.
Some of the participants at the sprint during demos.


Two themes emerged at the sprint around common pain points. Encouragingly, we (the developers among us) were able to create prototypes that start to address those pain points.  

Different math formats result in tedious re-work. First, we realized that a substantial number of education institutions and one major OER partner have been using Pressbooks with the BCcampus textbook extension to adapt the OpenStax textbooks. However, because Pressbooks and OpenStax use different math formats, if the textbooks have mathematics in them, after import, the math has to be hand recreated which is very time consuming.

We need a simple visual math editor, with a LaTeX-editing fallback for complex cases: Secondly, although there are many individual math editing tools, there is not a simple, easy to use math editor (that will also support advanced features) for the web that can be plugged into different tools, and that can produce the right math output for the plugged in environment.

Here is what we did during the sprint

  • Because participants had experience with a wide variety of editing tools and math conversion tools, we spent the first part of the sprint demoing a wide variety of tools and processes to create and adapt textbooks that have mathematics within them.
  • Then we generated an extensive list of "pain points" within these processes.
  • Next we generated a set of users stories from the points of view of three different users: faculty adapting and customizing textbooks, students answering homework problems for scientific and mathematical subjects, and professional teams copy editing and maintaining open textbooks.
  • From those we generated an extensive list of ideas of things that we could realistically do together at the sprint and tied those to the user stories they could serve.
  • Two technical themes emerged and the developers divided into
    • Team A that would concentrate on getting textbooks from one of three editing environments represented at the sprint (OpenStaxCNX, Pressbooks, Manuscripts), and especially solving the problem of getting OpenStax math format converted to Pressbooks math format.
    • Team B worked on an editor widget for writing equations visually or using LaTeX and then getting them back into a document as MathML, LaTeX, or an image.
  • Both teams were composing existing tools, not writing things from scratch, which is one of the fantastic results of opensource software. More details to come.
  • Each day we did demos and retrospectives from the sprint.
The next blog post will have more information about the prototypes that were created with links to demonstrations and source.


  • OpenStax ( - Kathi Fletcher, Phil Schatz (, Ross Reedstrom, Dante Soares, Ryan Stickney. The OpenStax team is interested in improving math editing for their internal textbook production and interested in making customization of the textbooks less cumbersome for organizations adapting the textbooks. Two of those organizations are here at the sprint.
  • OERPUB ( - Marvin Reimer is an experienced developer who worked with Kathi during her Shuttleworth Foundation (SF) fellowship ( and works with and Marvin wrote a google docs, latex, etc converter that also publishes to OpenStaxCNX.
  • Katalyst Education (  Christopher Sweeney, Tomasz Stach, Wojciech Ludwin, Krzysztof MÄ™drzycki, Iris Gau, Michael Moran. Katalyst Education has been working with OpenStax on Internationalizing the OpenStax user interface (OpenStaxCNX) and will also be publishing free college textbooks in Polish. They are helping with OpenStax’s efforts to create online versions of the textbooks that have the same numbering and collation as the PDF versions of the books, and they are committed to developing an easy tool for authors and editors who wish to adapt OpenStax textbooks.
  • BCcampus - Lauri Aesoph and Brad Payne (remote). BCcampus has been managing the B.C. Open Textbook Project since this project was announced by British Columbia Ministry of Advanced Education in 2012. Brad Payne, Senior Technical Analyst, developed the Pressbooks Textbook plugin for Pressbooks and provides technical support for and continuing development of this and the B.C. Open Textbook Project. Lauri Aesoph, Manager of Open Education, manages the ongoing effort to import all OpenStax textbooks into Pressbooks to allow easier adaptation of these books by faculty in B.C. and elsewhere.
  • Matias Piipari of - developer of the scholarly authoring tool Manuscripts, which includes MathJax based math rendering, an equation editor, and ability to convert between math formats on importing and exporting documents.
  • Omar Al-Ithawi - Software Engineer at  Edraak, an Arabic MOOC platform based in Jordan. Omar recently released a MathJax extension for Arabic and RTL.

Wednesday, February 12, 2014

Linking to Objectives in the OERPUB editor (a prototype between MIT OEIT folks and OERPUB)

Decorative, colorful concept map
Learning Objectives, Concept Maps
Image: By Sborcherding at en.wikibooks
[Public domain],
from Wikimedia Commons
The exploration: When creating textbooks and interactive learning activities, wouldn't it be cool if authors (and eventually others) could easily link material to learning objectives? This is the second exploration that OERPUB, Lumen Learning, and MIT's Office of Educational Innovation and Technology (OEIT) took on together in Salt Lake City. Linking materials (textbook, activities, videos, quizzes) to learning objectives makes them easier to find, and could also allow navigation by objective rather than by a single linear path through the material.

The Scenario: An author is writing a textbook or course in the OERPUB editor. Perhaps it is a physics course, and the course has a set of objectives that it teaches (or hopes to). The author is writing a section on lattices and the ways that x-rays scatter through crystalline structures. Since the physics department at MIT has defined this as a learning objective, it would be great if the author could easily specify that a reading teaches this objective.

The Components: MIT's OEIT has a service for storing and looking up learning objectives, called MC3. MC3 has an API for returning learning objectives. Before we got together, Cole Shaw took the OERPUB editor and embedded it in a page that connects with the MC3 server. The screenshots below show his prototype. He added a new "widget" to the editor for adding an activity and wired it up to include an objectives drop down. The choices in the drop down are coming from the MIT's objectives server. He copied an existing widget and modified it.

shows the editor with a drop down added to choose which server to get objectives from and which set of objectives to use.
Cole added a top toolbar for choosing where objectives
should be looked up.

Shows a drop down "Bragg's Law Outcom 3B1" is chosen from amoung a set of options.
Here is the drop down in an activity added to the document. The choices
are looked up live. Once one is chosen, it is added to the activity.

And then when we all got together, Cole and Tom Wooward worked together to take Cole's work and make it a widget that works in the github-bookeditor. That is shown below. Tom also showed Cole some of the ways to configure educational widgets within the editor. (That also tells us where we need to improve documentation for developers.)
Shows a "Read"ing activity, with "Bragg's Law" chosen as the objective
This is the same widget, but in the github-bookeditor. The
server to query is hard-coded. This will live on a branch
to show how such a thing can be done.

Really making this kind of thing widely useful for general users of the editor, requires more thought, time, and effort. MIT is hosting their own course objectives, and their software provides the store and lookup service. But these aren't general purpose. The user interface would need to provide ways of configuring which objectives are relevant, etc.

If we did come up with a way to do something like this, I would love to see a way to make choosing an objective a standard option on all content sections and educational widgets. In other words, an author could attach an objective to essentially anything within the HTML and the editor would provide an easy UI for doing that and a simple encoding as metadata to store in the document. I think that would probably be's educationalAlignment.   

Technical notes and links:

Tuesday, February 11, 2014

Sprinting to embed assessments and learning objectives with MIT, Lumen Learning, and OpenAssessment

Shows a quiz in a textbook page too small to see
Idea: Wouldn't it be cool to have a really easy way to embed interactive assessments in textbooks, epubs, and courses?

People: The folks I have just been meeting with thought so and we got together to explore a few prototypes. I have been in Salt Lake City working in the Marriott Library at University of Utah, hosted by David Wiley of Lumen Learning, and joined by Brandon Muramatsu and Cole Shaw of MIT's Office of Education Innovation and Technology (OEIT), Justin Ball and colleague James from Atomic Jolt consulting, and Tom Woodward of OERPUB via Daft Labs.

Scenario: The following scenario sets up our first exploration. Lumen Learning is adapting a biology textbook from Open Stax College. They are creating courseware for college faculty that takes each section and adds interactive, formative assessments, and discussions and analytics and other cool stuff. They are creating completely open banks of questions to go along with the books and these will live at Open Assessments is building a quiz player that works like a youtube video player. You find a quiz you like and use a simple embed code to include that anywhere you want.

Exploration: So what we wanted to explore was including the ability to find and add a quiz from Open Assessments in the OERPUB editor. So, imagine you are creating a textbook section, or a learning activity for college biology and you have just written the section on parts of the cell, and you want to help students retain what they have learned. So you click on the 'quiz' button in the editor, and search for quizzes about cells, preview the quiz, and pop it in. This is what we put together yesterday. Keep in mind this is code written quickly to see how to do this kind of thing while we had all the experts together. It isn't polished and beautiful. But the impressive thing is that we got this done in a couple of hours. The following screen shots show what we did.

"Insert an Assessment", with two fields, "Assessment url", or "Search"
After clicking on the quiz widget in the editor, search
for "cells".

"Insert an Assessment", with "Search" field filled in with "cells"
The search uses's API,
and returns one result. Click on "select" to preview it.
Shows "Question 2 of 3" about parts of a cell, with "golgi body" selected as the answer.
Preview the assessment to make sure it is what you want. The
preview is live, so you can check the answers and all the
questions in the quiz.

The quiz is embedded in the content and will play in the editor
and also in the textbook as long as there is an internet
connection. The quiz is being played by
The actual quiz is stored as a qti file at openassessments.

Technical notes and links:
Upcoming posts

Thursday, January 23, 2014

Once upon a time textbooks were hard to create ...

My Shuttleworth Foundation Fellow colleague, Arthur Atwell, sent an intriguing challenge out to our gang of fellows. The challenge was to come up with a pitch for our projects that follows the Pixar style of pitch, as described in Daniel Pink's book, To Sell is Human (see full reference at the bottom of the blog entry). The beauty of the style is that it really emphasizes story, which of course is at the heart of movies, and really is at the heart of all human endeavor. But it isn't always easy to articulate the importance and vision of a technical software project. At least not for those of us who regularly geek out and focus deeply on technical things.

The pixar style has the following components:

Once upon a time, ...
Every day, ...
One day ...
Because of that, ...
Because of that, ...
Until finally...

So here goes. Here is my story of the vision behind the work I have done as a Shuttleworth Fellow. 

OERPUB Movie-Pitch
Once upon a time, textbooks were hard to create, expensive to buy, and out of date within a short time.

Every day, college students paid $150 for an algebra book containing information that is hundreds of years old. High school students learned from ten year old Biology textbooks, authors struggled to make everything look good and cursed while they tried to edit math.  Nobody could use the content in the textbooks to create interactive flashcards or quizzes.

One day we created a textbook editor that is easy to use and saves books to github (a place for freely storing books and software). We made sure the hard stuff, like editing mathematics, formatting the books, and delivering them to students was actually easy. And we made sure that things like definitions and homework problems were easy to reuse.

Because of that, authors can collaborate to build textbooks, deliver them to students online, on mobile devices or in print. They can make updates immediately, and share textbooks with others for translation and adaptation. Software developers can create interactive flashcards and study tools that use the content from the textbooks.

Because of that, textbooks are a pleasure to create, cheap or free to buy, always up to date, and part of a much more interactive and engaging experience.

Until finally we've transformed textbooks into true engines of learning.
Reference: Pink, Daniel H (2013-02-07). To Sell is Human: The Surprising Truth About Persuading, Convincing, and Influencing Others (pp. 172-173). Canongate Books Ltd. Kindle Edition.

Friday, January 10, 2014

Recent talks about creating, editing, and remixing textbooks with the OERPUB editor

Normally I would create a post after each talk, but I got behind so I am going to link in my talks from "The New Publishing" W3C workshop, Books in Browsers, and Open Ed, all in fall/winter 2013. 

First up in September was the W3C Workshop, The New Publishing and the Open Web Platform. The title for my paper is practically its own abstract, "Semantic HTML5 is the Future of Textbook Publishing and Non-technical Authors Can Participate using Customized Web Editors that Support Accessible Authoring".  In the paper, I argue that we should be writing textbooks in HTML5 using a clean, open, and semantic format, so that books can be read online, on the web, and in print, and more importantly are easy to keep up to date, combine, translate, and make accessible to learners with disabilities.

Next up in October was Books in Browsers,
My talk, "Textbooks in Browsers: An Editor for Creating, Adapting, and Sharing", covered our open-source editor for textbook authoring that lets authors create, adapt, and remix textbooks that display well in the browser, on mobile devices, and in print. Since the editor itself runs in a browser, and the book can be read on your browser, it was a perfect fit for the conference. The slides (linked above) show how the editor supports mathematics, accessible images and tables, and structured features like definitions and exercises, using a constrained subset of HTML5. At the end, the slides give links to use the editor and for developers to get involved. You can see me giving the talk, here (minute 8:40 to 28:17). My favorite tweet during the talk: "Github-Bookeditor!? Yes, it's a thing. A very awesome idea brought to life by @oerpub #bib13"

In November, I spoke at Open Ed 13, on "Write to share; Real remix realized". Remix is the gold-standard of OER effectiveness, but technical barriers have made it hard to do, even when author-educators want to share their content and reuse and adapt high quality open resources. OERPUB's open-source editor solves this problem by making it easy for authors to create rich open textbooks that can be remixed and shared. The editor supports editing mathematics, embedding multimedia (coming soon), and is supportive of creating content that is accessible to learners with special needs. I reported on Adrian Garcia's research on best practices for motivating author-educators to create semantically rich OER that is easy to share and remix. He found that K-12 teachers were especially interested in content that will work for learners wtih disabilities. I also reported on our textbook creation sprint in South African with St. John's College teachers (see more in this blog entry). We got great feedback from the teacher, enthusiastic support for the collaboration and drag and drop features, and plans for custom Physics and Chemistry textbooks for 11th and 12th grades coming out over the next year. They are using Siyavula textbooks, Open Stax College textbooks, and their own materials. ( You can see the video of me giving the Open Ed talk here).

Tuesday, January 7, 2014

Video plugin prototype (from last year) and upcoming implementation plans

Apparently, I never blogged about the prototype video plugin that two OERPUB interns created for the Aloha-Editor last year. We are getting ready to add multimedia capability to the github-bookeditor and so I was looking for that blog entry without success. So better late than never, here is a link to see how the prototype worked.
the editor with the video chooser dialog open
Screen capture from a screencast of the video plugin in action.
Click on the image to run a video of the process, or click here
I like how the prototype lets authors search for videos and pick them from a list that includes a thumbnail and description. There is always a URL backup, but the search means that authors don't have to leave and find the video and cut and paste in a link.

The student developer interns, Max Grossman, and Gbenga Badipe, worked together to create this prototype and explored the possibilities using the Youtube, Vimeo, and Slideshare APIs. They have long since graduated and started computer science careers, but their work lives on.

We are planning to add a plugin soon to the editor so that authors can include video and slides. We will be working with our friends in the accessibility community to make sure that we make it easy for authors to include information about audio and transcripts so learners find content appropriate to their needs. More coming on this topic.