Computer Science for All – Are We Asking the Right Questions?

Every two weeks, the CSTA K-8 task group hosts a twitter chat using the hashtag #CSK8. These twitter chats help teachers like me connect with other computer science education enthusiasts; they offer us a place to share and learn new ideas for our classrooms. Since I am part of the CSTA task force that hosts these chats, I have learned a lot on how to run these twitter events. Picking the right questions for a chat is key to its success.  The questions must provide the right amount of structure and be interesting so that all participants contribute to the chat. This is difficult since we do not know who will actually join the chat.
The chat on Feb 10 was on the new initiative proposed by President Obama called ‘Computer Science For All’ (To read more on this initiative, see  “CS for All”  and  Watch the president’s full remarks here). The chat was moderated by another CSTA Task force member Vicky Sedgwick and myself.
Computer Science For All initiative is still in the early phase, with of course no clear idea on whether it will ever be approved. However, the initiative has opened up the discussion on computer science access to a wider audience, and it was a perfect topic for our chat. Vicky and myself struggled to come up up with the best set of questions and we were modifying them as the chat progressed.   Selecting the right questions to ask on this topic helped us think more on the big question – how do we really provide computer science education to all? Here are the questions that we finally used on this chat. Take a look below –  what would be your answers?
  • Q1: Obama said “we have to make sure all our kids are equipped for the jobs of the future.” Is this really why #CSforALL is needed? #csk8
  • Q2: Why should #CSForAll be a federal initiative? Can’t we just rely on state/local/industry/non-profit efforts? Pros/Cons? #csk8
  • Q3:  How will we find teachers for #CSForAll & what is needed in terms of professional development & teacher credential programs? #csK8
  • Q4: There are currently many states with their own CS standards & more writing them. How does this affect #CSforAll or does it? #csk8
  • Q5: How would you spend $4 billion? What is most important? K-8/High School/PD/In-school programs/AfterSchool programs/Diversity? #csk8
If you are interested in reading the conversations on this topic (or other topics), check out the archives of the #CSK8 chats at the CSTA K8 G+ community at
Do you think we are asking the right questions? If you had five questions to ask on this topic, what would they be?

2016: The Year of CS Education

A Prediction Comes True…

When asked for a New Year’s prediction a few weeks ago, I responded that 2016 would be the Year of Computer Science Education.  I did not anticipate just how accurate that prediction would turn out to be just 30 days later.  And it appears that we are just getting started, thanks to the incredible support and commitment of the White House and this Administration on behalf of CS education and CS teachers.

CS education is about students.  On January 12, as he began to speak to national priorities, President Barack Obama led with CS Education.  He said that, “In the coming years, we should build on that progress, by … offering every student the hands-on computer science and math classes that make them job-ready on day one.”  As Executive Director for one of the first CS teacher member organizations, it was an exciting moment to hear the President lead off with a statement so aligned to our members’ profession.

CS education is about access.  On January 20, the White House announced the Champions of Change for Computer Science Education. I was thrilled to see recipients like Jane Margolis whose book, Stuck in the Shallow End: Education, Race and Computing, motivated me to pursue this position several months ago.  The recipients of the honor included a diverse and deserving collection of individuals working to improve access to computer science education.

CS education is about collaboration.  Then today, January 30, I was again both excited and awed, as the White House announced the Computer Science for All initiative (#CSForAll)—the President’s plan to give all students across the country the chance to learn computer science in school.  It is a plan with aggressive goals, bipartisan support, and multifaceted commitments from an amazing array of participants spanning federal and state agencies, corporations, non-profit organizations and academic institutions, school districts, and teachers.

CS education is about teachers.  It is clear that many more exciting announcements are to come.  On behalf of the Computer Science Teachers Association (CSTA) and the teachers it represents, I thank the Administration for its “above and beyond support” for CS education and recognizing that providing access to quality CS education to all students requires developing and supporting CS teachers.  I am also appreciative to the Administration for creating mechanisms to enable CSTA to actively participate and engage in the events leading up to today’s announcement.   CSTA is excited to be involved and contributing to this collaborative effort.

…And CS Education is Just Getting Started.

CSTA recently developed a new 10-year vision, supported by the first of three strategic plans.  The themes of students, access, collaboration, and teachers underpin that framework.  For the next three years our primary efforts will focus on teacher professional development, programs related to our big IDEA (Inclusion, Diversity, Equity, and Access), and maturing our association practices.  These three priorities are supported by a set of five strategic levers and a range of specific measures and activities.

As part of CSTA’s commitment to #CSForALL, we will pursue and implement a new professional development (PD) model for CS Teachers that includes:

  • A developmental assessment with personalized roadmap to help teachers focus PD on skill development needs and programs that could address those needs.
  • Hybrid (online + in person) PD experiences to increase access to PD for teachers.
  • A digital portfolio or digital badging model to enable competency-based micro-credentialing.  This provides a means for teachers to demonstrate CS skills and track their progress toward a master-CS teacher status.

We are on track to pilot some of the above elements as early as this spring.

This year CSTA will establish a Diversity Educational Leadership Program (DELP).  DELP will provide PD to cohorts of teacher-leaders coming from diverse and underrepresented backgrounds in CS.  The goals of DELP are to improve access to leadership and development opportunities for underrepresented teacher segments, support a growing network of effective teacher-leaders and CS advocates in their classrooms and communities, and increase the visible pool of diverse candidates for leadership positions in CSTA and other K-12 CS organizations.

CSTA is also stepping up its own capabilities, such as going live with the “alpha” version of our new member management system this past week.  In addition to a new website that is mobile-friendly, and easier to navigate and update, we will have tools to enable more members to engage and volunteer in activities of the association.  There will be new tools to support chapters.  New tools to support advocacy or outreach among segments of members. There will be new ways for members to communicate with each other and new resources to help make #CSForAll a reality.

Later this spring CSTA will unveil new branding, as we evolve into – the member organization for K-12 computer science teachers. With 22,000 members across 130 countries, with 62 local member chapters, and as founding partners of other CS educational organizations, like, NCWIT, and TeachCS, we will continue to seek out and engage in opportunities to collaborate that include CS teachers and further enable access to quality computer science education for all students.

Getting Engaged in the Future of K-12 CS Education

These and many of our other planned initiatives, such as a series of PSAs and content to promote awareness and understanding of what CS is, link back to the themes and priorities identified by the White House as part of #CSForAll:  Students, Access, Collaboration, and Teachers. Getting there will require innovation, entrepreneurship, collaboration and support from a great variety of organizations and individuals.  CSTA greatly appreciates the work of this Administration which has elevated CS education and the needs of CS teachers to a national priority.  We look forward to the great works that will come out of the current #CSForAll commitments, and for those that will follow.

2016 is going to be a great year for K-12 CS Education.  Please keep following #CSForAll and #CSTA on Twitter for more developments or reach out to CSTA if you are a CS teacher or organization who would like to be involved in our evolution.

About CSTA:  The Computer Science Teacher’s Association (CSTA) is a member-based organization founded in 2004 by ACM, the world’s largest educational and scientific computing society.  CSTA’s mission is to empower, engage, and advocate for K-12 CS teachers worldwide.

Are You Ready for Computer Science Education Week?

As I write this blog post, I am in the middle of planning for Computer Science Education Week. I have prepared my Eventbrite site ( and have emailed the link to the principals of the neighboring schools. This year will be the third year that my computer science students and I have hosted a community Hour of Code event at the high school for elementary and junior high students.

Last year I was so overwhelmed with the number of attendees, I have decided to move the event from the two adjoining computer labs to the library where I can better manage a large group. I will have stations for each age group and a marker station where the students will create a light-up Christmas card. Laura Blankenship (CSTA board member) helped me with some suggestions. I also found on line the directions for a Christmas tree card ( )that I plan on using.

I am also planning on setting up three tables with old computers that can be taken apart. I will have index cards with pictures of the main parts and an explanation of their purpose. My students will be preparing the cards.

Finally, there will be stations setup by age groups for students to use the tutorials ( The Computer Science students are looking forward to helping with the event.

For the high school student event, I have a local college student who will come to class to discuss her internship at an aerospace company and her experience as a college student. I have also requested skype calls from professionals in the computing industry at I had two speakers last year that worked for Microsoft and they were so patient with my students! I will also play the morning announcement created by CSTA on Monday, December 7, announcing Computer Science Education Week ( On the same webpage are video announcements if you have that feature available at your school.

I have another Hour of Code event planned for lunch for students on campus with my Computer Science students assisting. Additionally, my school district will be honoring Computer Science students at the school board meeting on December 8 in honor of Computer Science Education Week.

I am looking forward to a busy but exciting week.

Please share your plans for Computer Science Education Week.

Myra Deister
CSTA At-Large Representative

Cybersecurity is everywhere. Is it in your K-12 CS program?

Scarcely a day goes by without the mention of cybersecurity in the news—from Edward Snowden breaching security at NSA (and now following NSA on Twitter), to customers of Target and Home Depot having their data compromised, to Hillary Clinton’s private email server and private email account while she was Secretary of State, to hacking of sensitive government data by foreign citizens, cybersecurity is in the news and is newsworthy. One of the more common themes in cybersecurity is the dearth of qualified cybersecurity professionals and how the United States might address that lack in the education system. Several colleges and universities have begun to add cybersecurity programs, and credentialing bodies are developing certificates and credentials for those already in the workforce but perhaps lacking the proper skills and training. The ACM Education Policy Committee and the ACM Education Council have both entered into discussions about cybersecurity education. The Committee for Computing Education in Community Colleges (CCECC) of the ACM Education Board has done quite a bit of work on cybersecurity education at the community college level. But, where is cybersecurity education in K-12?

Once again I had the pleasure of representing CSTA at a great K-12 education conference. This conference was the National K-12 Cybersecurity Education Conference held in Linthicum, Maryland (near Baltimore). Tammy Pirmann (CSTA School District Representative) and I both presented sessions about CSTA and cybersecurity education. Tammy’s panel discussion focused on curriculum and programs of study while my panel discussion and break-out session focused on standards. The conference had a great mix of K-12 educators, post-secondary educators, state-level educators, industry and governmental organization representatives, and curriculum developers.

There are some really awesome cybersecurity initiatives taking place in K-12 classrooms. I was able to participate in three hour workshop about the Baltimore County CyberSTEM program in which elementary school students engage in hands-on activities and learn about the field of cybersecurity. They learn to apply basic security concepts through gaming, modeling and simulation, robotics, digital forensics, cryptography, system vulnerabilities and cyberethics, safety and security, while investigating exciting careers that interconnect the fields of science, math, technology and computer security.

The sessions about the standards pointed out that there is much common ground among the organizations working in the computer science, engineering, and technology education space. It was good to have an open dialog and to be reminded that we can all work together to become stronger—and to promote cybersecurity education through our various groups. Even very young (Pre-K) children need to learn the basics of cyber safety—most of them use some sort of device before they ever enter a formal classroom. There was much discussion about standards and where cybersecurity education standards would logically fit. The National Initiative for Cybersecurity Education (NICE) is working with the Department of Education and the National Science Foundation to lead the Formal Cybersecurity Education Component.  Their mission is to bolster formal cybersecurity education programs encompassing kindergarten through 12th grade, higher education and vocational programs, with a focus on the science, technology, engineering and math disciplines to provide a pipeline of skilled workers for the private sector and government. They have a great interactive Cybereducation map that highlights schools, teachers, companies and agencies that support Cyber Education in the United States.

The good news for K-12 CS educators is that there are many great resources to assist you in the classroom as well as opportunities for you and your students. The National CyberWatch Center K-12 Program extends the National CyberWatch Center mission of advancing cybersecurity education by leading collaborative efforts and strengthening the national cybersecurity workforce to the K-12 community. Check out their fantastic summer programs for students (camps) and PD opportunities for teachers. Perhaps you attended a program last summer—we learned about them at the 2015 CSTA Annual Conference! (They also have curriculum resources for teachers.)

So, when you are ready to implement cybersecurity education in your K-12 CS classroom, the resources are there. You will be able to find curriculum resources, PD for yourself, speakers for your students, summer camps for your students, career opportunity resources, and much more. You might even plan on attending the next National K-12 Cybersecurity Education Conference. Cybersecurity is everywhere, and it should be in your classroom.

Deborah Seehorn
CSTA Board of Directors Past Chair

Website Links:

National K-12 Cybersecurity Education Conference

CyberSTEM Program

National Initiative for Cybersecurity Education (NICE)

Interactive Cybereducation Map

National CyberWatch Center K-12 Program

CSTA Annual Conference

Advocating for CS Education – Strategies from Connecticut

By: Chinma Uche

The November Voice is full of great advocacy ideas that you can use in your chapter. Be sure to check it out! (

Connecticut CSTA (CTCSTA) members work hard year-round to shine a bright light on CS across the state and to provide opportunities for students. Real progress and change requires many strategies and persistence. We hope that this list of our focus areas will inspire you to create a plan to advocate for CS education in your school, district, and state.

  • CSEdWeek Activities: CTCSTA starts planning CSEdweek activities during the first meeting of the school year. Ideas are shared and every CTCSTA member commits to plan and execute at least one activity. At the very least, all members participate in the Hour of Code.
  • Member resources: We have used the opportunities provided to two of our members as K5 Affliates to introduce CS into elementary schools.
  • Government:
    • We apply to the Governor’s Office in late October for an official statement in support of CSEdWeek. Last year, our members met with members of the Connecticut General Assembly (CGA) and were fortunate to get a bill written to support CS in Connecticut. Our members made presentations to the Education Committee of the CGA and the bill passed and was signed into law on June 23, 2015. Because of the passage of Public Act 15-94 (, Connecticut schools are required to introduce CS by the next school year.
    • We also garnered support from the Connecticut State Department of Education, where five of our members serve in the newly created Computer Science Advisory Committee. This committee is exploring ways to bring CS to all Connecticut schools (
  • Curriculum: The Mobile CSP project, funded by NSF and led by Professor Ralph Morelli of Trinity College, has expanded access to CS in many high schools. The curriculum has been very widely received in Connecticut and Mobile CSP teachers are strong advocates for CS in other schools and districts.
  • Partnerships: CTCSTA has a strong relationship with the Connecticut Education Association (CEA) since many of our members belong to the CEA. CEA supports bringing CS to all Connecticut schools and blogged about Mobile CSP training (org/2014/08/25/students-teachers-and-school-districts-benefit-from-computer-science-professional-development/) to their 43,000 members. In the past, many CTCSTA members have planned activities with community organizations such as the Girl Scouts. During CSEdWeek 2014, we jointly hosted a “Women in STEM-C” event that brought together about 100 young girls to enjoy STEM activities and create apps. The Lieutenant Governor attended the event in support of increased access and diversity in computer science ( We continue to reach out to schools of education such as the NEAG School of Education at the University of Connecticut and hope to form a relationship that will help them include CS in their teacher education program. Other partnerships include:
  • CTCSTA supported the Connecticut Science Center in organizing #BeautyByMe Girls-Only Hackathon (
  • CTCSTA supported the YouMedia Group at Hartford Library in the development of their CS program (
  • CTCSTA worked with AAUW ( to host a CS program for girls.
  • CTCSTA is working to bring the Technovation Challenge to more Connecticut girls. (
  • The number of girls receiving the NCWIT Aspirations Award continues to increase in Connecticut because of the work of CTCSTA members.
  • CTCSTA members support the work of Random Hacks of Kindness Jr. (

Our experience shows that CS advocacy benefits teachers as well as students. CTCSTA members who engage in advocacy in their districts are recognized as leaders. Jackie Corricelli (Conard High School), who led her entire school (1500 students) in the Hour of Code, received $10,000 from for her class. She also received the Presidential Award for Excellence in Math and Science Teaching. Melissa Fearrington (Simsbury High School) trained elementary school teachers in her district to introduce the curriculum (Courses 1-3) to their students and was recognized as the Teacher of the Year.


CSEdWeek: Ideas to Fit Your Schedule

The November Voice is full of great advocacy ideas. Be sure to check it out! (

CSEdWeek is quickly approaching and it’s likely that you already have some great projects planned for the week of December 7–13. But if not, here are a few ideas, small to large, to fit the time you have.

You will find many more ideas and resources at,, and Great projects can be used any time of the year so don’t limit your computer science (CS) education advocacy ideas to just one week in December.

Pledge your support for CSEdWeek and start planning how you will fuel the future with CS education at

15 Minutes:

30 Minutes: 

  • Assign students to ask their families to spend an entire day without using any computing technology (including no car, microwave, or digital television) and record their experiences. Discuss their findings in class the next day and relate to CS careers.
  • Prepare a showcase of student computing projects.
  • Offer a lunch break “CS Escape” once a week or once a month to invite non-CS students to “play” with CS concepts and tools such as CSUnplugged, Scratch, AppInventor, or Kodu.
  • Schedule your students to take turns in the cafeteria or student commons to demonstrate cool CS projects from your class.
  • Learn a few “magic tricks” to spark up your introduction to CS concepts.
  • Redecorate your classroom with new posters among the many available from CSTA or

1 Hour: 

  • Participate in Hour of Code.
  • Invite former students who are college CS majors to visit your classroom to tell about the exciting things they are learning. 
  • Assign cross-curricular projects. Ask other teachers to allow your student to “show off” their computing version of the assignment.
  • Submit proposals to share your innovative CS teaching strategies at conferences.
  • Encourage your CS students to host an open house for other students.
  • Contact your local government representative to ask for support in having December 8–14 proclaimed CSEdWeek.
  • Engage students in playing a matching game of inspiring CS quotes to famous people. Expand into an assignment on CS careers.
  • Explore the resources from the Computer Science Collaboration Project to find Exemplary Practices for Engaging Hispanic/Latino(a) Youth in CS ideas.

3+ Hours: 

  • Arrange a field trip to a local high-tech company or corporate IT department; invite parents also.
  • Plan CS projects that involve “social causes.” Recruit community members to serve as “clients” for your students.
  • Host a parents’ night that showcases the fun, exciting, and meaningful career opportunities in computing and debunks the myths about the dwindled IT job market.
  • Plan a CS exploration day for potential students and parents. Gather ideas from programs such as Computer Mania.
  • Look for technology contests to showcase your students’ ideas such as the ACSL, Google Code-in, Aspirations in Computing, Imagine Cup and others.,,,
  • Investigate ways to engage underrepresented students in computing.
  • Form an advisory team of students, parents, other teachers and administrators, business leaders, and others to plan CS promotion projects in your community year round.




AccessCS10k: An Introduction to the Quorum Programming Language and Evidence-Oriented Programming

Andreas Stefik, Ph.D.
Assistant Professor, Department of Computer Science
University of Nevada, Las Vegas

Computer programming in K-12 education has become increasingly mainstream in the U.S., especially with the excellent work being conducted by groups like, Project Lead the Way, CSTA, and many others. These trends have been happening for many reasons, not the least of which being a growing awareness of competition from abroad and the well known shortage of professionals in the field (a problem we are acutely aware of in Las Vegas). In this blog post, I will discuss work on the Quorum programming language, the first, so-called, evidence-oriented programming language. By our current count, Quorum is taught in approximately 34 K-12 schools in the U.S. From surveys, we estimate over 1,800 students will be taught Quorum next year.

In this blog, I’ll outline the origins of the Quorum project and evidence-oriented programming in general. More information can be found about Quorum at our website
( Quorum has, to our knowledge, the first Hour of Code that is accessible through a screen reading device and a talk on the language was recently presented at Strange Loop and is available here:

Research on blindness and the origin of Evidence-Oriented Programming

The Quorum project began in approximately 2006 with an exploration I was conducting into the experiences of computer programmers who are blind or visually impaired. Initial interviews and discussions with professional blind programmers made it relatively obvious how many challenges the community faced. In many cases, mainstream tools (e.g., Visual Studio, NetBeans) were not particularly well suited, or in some cases did not work, for this community. More crucially, after discussing the issue with blind professionals, I suspected the challenges for K-12 students were likely worse, as learning Braille or a screen reader can be challenging for blind children. Consider for a moment why learning a programming language like C, on top of learning screen reading skills, might be difficult. While most individuals might see syntax like: for(int i = 0; i < 10; i++) {}

and be fine with it, this is typically translated into speech for the blind. While there is no perfect way to write this, readers are encouraged to say out loud “for left paren int i equals zero semicolon i less than ten semicolon i plus plus right paren left brace right brace.” Similarly, compiler errors when translated to audio can be difficult to understand. A trivial error (e.g., missing a variable definition in C++), might take a minute or two to “listen” to. Mainstream students have difficulty with these things too, but they are exacerbated in students with disabilities.

After making these observations, I concluded that one obvious way to approach making programming easier was to improve the tools (e.g., talking debuggers, accessible editor hints, accessible completion, accessible code folding). I invented all of those things (most are in Sodbeans, a tool for the blind used throughout the U.S. today), and they do seem to help, but C itself is still hard to “listen to.” Ultimately I couldn’t get this nagging question out of my head, “What was the evidence for the design of C to begin with?” So I began to investigate, but weirdly … I could find little human factors evidence in the academic literature.

Historical Context on Evidence Standards:

A slight detour is necessary to help readers who are not familiar with historical evidence gathering techniques. Giving credit where due, I highly recommend reading a paper by Ted Kaptchuk on the history of blind assessment, which influenced my thinking:

Let’s briefly discuss this history. First, it was not always the case that medical practitioners, psychologists, and others used experiments in science. In medicine, the techniques started largely in the late 18th century after Louis XVI created a commission to study the bogus theory of animal magnetism (mesmerism). Benjamin Franklin, the head of the commission, used the idea of “sham treatments” to uncover problems with this (bogus) theory.

As science progressed, techniques became increasingly rigorous. Scientists used primitive placebo tests as early as 1834, although they still had no concept of experimental design. By the late 19th century, psychology was developing and we start to see the use of randomization, at first to explore sensory perception and to evaluate (bogus) supernatural claims like those made by psychics (e.g., talking to the dead).

Experimentation moved forward again in pharmacology, in part because of Charles Edouard Brown-Sequard’s claims that testicular extract from guinea pigs would “rejuvenate mental and physical health.” The scientific community, now increasingly skeptical of claims made without evidence, started still rather primitive assessments within a few months. Finally, by the mid-1930s, we obtained what many scientists think of today as the randomized controlled trial, made possible largely by Ronald Fisher in his famous, “The Design of Experiments.” While not mentioned by Kaptchuk, experimental design today has made other important leaps forward, most of which are unfortunately not internalized in computer science like they are in other disciplines (e.g., studies on replication, registered randomized controlled trials). Walter Tichy has probably discussed this more than most.

Evidence-Oriented Programming

Back to blindness and programming, consider observations that children who are blind had extraordinary difficulties with programming languages, and also that the academic literature was sparse with human factors data on programming languages. Certainly, there exists some papers on the topic (e.g., ESP, PPIG, Plateau, a small number of trials in the 70s and 80s), but the number of randomized controlled trials is small. Programming languages make up the entire foundation of modern software — effectively every computing device that everyone owns is built with one. Yet, the top conferences in programming languages used rigorous proofs to determine if features worked, and had significant empirical data to evaluate performance, but used anecdotes for human factors. I didn’t understand why this would be (yet), but by 2009, we started investigating ourselves, largely in this paper:

Stefik and E. Gellenbeck. Empirical studies on programming language stimuli. Software Quality Journal, 19(1):65-99, 2011. 10.1007/s11219-010-9106-7.

Some problems became obvious. Surveys showed words like “for, while, or foreach” were, in a bizarre and unexpected twist of irony, the three least intuitive choices for people in our sample (which we later replicated). This is ironic because these choices are common across a large number of programming languages.

Surveys only tell us so much though, so we went further, which is to take from the medical playbook, using sham (or dummy) treatments. In the first test, we used what I called a “Placebo Language,” basically a randomly designed programming language where symbols are chosen from the ASCII table (one could imagine other placebo languages). We sent the first of these tests to a small workshop called Plateau in 2011:

Andreas Stefik, Susanna Siebert, Melissa Stefik, Kim Slattery. An Empirical Comparison of the Accuracy Rates of Novices using the Quorum, Perl, and Randomo Programming Languages. Workshop on the Evaluation and Usability of Programming Languages and Tools (PLATEAU 2011). Portland, OR, October 24th, 2011.

The result, at least to me, was an eye opener. Perl, despite the fact that it was a very popular programming language with significant community support, was apparently not detectably easier to use for novices than a language that my student at the time, Susanna Kiwala (formerly Siebert), created by essentially rolling dice and picking (ridiculous) symbols at random. That result was astonishing and I frankly wasn’t sure I believed it. So, we ran a replication with three additional languages. The result was the same, replicating with extreme accuracy on a new sample (Perl within less than a percent). That work was published here:

Andreas Stefik and Susanna Siebert. 2013. An Empirical Investigation into Programming Language Syntax. ACM Transactions on Computing Education 13, 4, Article 19 (November 2013), 40 pages.

Interestingly, in this new study, the same result we saw with Perl we also observed with Java, a programming language so popular it is used in the Computer Science A – AP test in high school. Ultimately, using a technique we call Token Accuracy Mapping, which is basically a way to figure out which tokens may have caused the problems, it appeared that C-style syntax was plausibly the culprit. At the same time, using this new technique, we found a variety of problems with Quorum and borrowed from languages where the evidence showed they had a better design (e.g., Ruby’s if statement design, with the exception of the equality syntax ==, was integrated into Quorum 1.7).

Again, this finding surprised us. At this point, while there is a lot more to learn, we knew there was a problem and our strong suspicion was that the language community had not been gathering rigorous evidence on human factors, basically in its entire history, which we have since confirmed is an unfortunate truth. We hypothesize that this lack of evidence may be one of the leading causes of the so-called “programming language wars,” which we discuss at length in the literature:

Andreas Stefik and Stefan Hanenberg. 2014. The Programming Language Wars: Questions and Responsibilities for the Programming Language Community. In Proceedings of the 2014 ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming & Software (Onward! 2014). ACM, New York, NY, USA, 283-299.

In other words, if language designers are not even gathering evidence on human factors, then it’s not surprising that they seem to disagree about their designs or that students might needlessly struggle. In essence, language designers prove that their features work, and show evidence that they are “fast,” but they never seem to ask, in a scientifically defensible way, “What is the impact on various kinds of people?”

Creating Quorum:

As these studies progressed over time, and as we worked more with various communities, my wife and I decided that if we wanted to program using a language that used human factors evidence as a guide, we would have to build it ourselves. So, we developed the first Evidence-Oriented Programming language, which we called Quorum. Here are the basic ideas of this paradigm as I see it today:

  1. Design features of a language must have corresponding evidence from appropriately constructed randomized controlled trials for as much as is possible
  2. The language authors must allow the external community to suggest changes according to the scientific method, inspired by lessons from history
  3. Evidence from other techniques (e.g., software repository mining, educational data) if gathered from humans and rigorous is allowed

Now to be clear, Evidence-Oriented Programming is a paradigm, not a language, and the scholar Stefan Hanenberg in Germany thought it up at basically the same time. He came to the same conclusions I did for different reasons, with different experiments, on a different part of the globe (his story on the topic is very different from mine and fascinating). Others are fleshing it out better than I have, especially the scholar Antti-Juhani Kaijanaho, who has created what is perhaps the most systematic review of the history of the programming language wars ever.  

More crucially, it is a paradigm because the evidence on various design decisions (which I won’t get into here, but have written about extensively) is tricky to understand and requires significant statistical expertise to grasp. It also appears that some design decisions have trade-offs, the most well known of which to-date is that whether functions in programming contain type information (e.g., an integer, a number) appears to improve productivity of those past the third year of experience in college, but has a small, but non-zero, negative impact on novices at around a freshman year of college (depending on where in a program it is).

In other words, I can imagine one language designer using evidence to optimize human factors for one group, while another optimizing for another, with both taking all known controlled trials into account and therefore having some similarities in their design. As a simple example, a block language designer hypothetically designing for a child’s toy or learning might use the word “repeat” for iteration, which the evidence supports, yet a text-based language designer might use the same word when designing for industrial scale robotics or a NASA rocket where blocks might be impractical and where efficiency of the language is crucial (or some other reason). This type of thinking could provide increased consistency amongst language designs in the years to come, with an increasingly stronger foundation of evidence.

In any case, evidence-oriented programming is not a panacea, nor does it imply we will eventually get to the unlikely idea of “one language to rule them all.” Plausibly, what we are observing today is a paradigm shift in language design, going from next to no evidence at all on human factors toward an unwillingness of young scholars like myself to accept anecdotes or prestige as fact, for a basically similar reason Benjamin Franklin didn’t in the 18th century. No one knows where the evidence-oriented movement will end up, perhaps with some complex hybrid of blocks or visualization, or maybe with different domains having various kinds of approaches, or maybe even partial standardization of some aspects of language design within a century (or two). Wherever it ends up, the goal is to follow the evidence wherever it leads in order to make the next generation of programming technologies just as practical at industry scale, while also being easier to understand and use.

While Quorum began as a toy exclusively to help blind children learn computer science, it has grown in unexpected ways. While it is still used across the U.S. at around half of schools for the blind and visually impaired, about half of teachers that came to the 6th annual Experience Programming in Quorum workshop in Vancouver, WA represented mainstream schools. Quorum is a Java Virtual Machine Language, so it will run on most machines, and has a variety of libraries, including those for creating computer games, LEGO robotics support, and more. Quorum is accessible for people with disabilities, including those with visual impairments, and supports a robust and accessible development environment based on Oracle’s NetBeans. Since the language is built on industrial strength technologies, it can be used not just in K-12, but potentially in college or industry as well, for effectively any general purpose application. Finally, Quorum is under the BSD license (free for any use), and all curriculum is free under the Creative Commons License. Our curriculum goes through continuous improvement by dedicated K-12 teachers and is being mapped to Computer Science Principles and, hopefully soon, the common core. More information can be found at:

Must-read (and Share) Report on US K-12 CS Education

They say you can have data without information, but not the other way around.  So for those of you seeking data on why CS in K-12 matters there is good news today.  There is a new report out from Google and Gallup, which I particularly encourage all of our members in the US to read and share.  Whether you are in K-12, higher education, government, or the corporate environment the data and information contained in this report is important.  As we head into another school year, it would be great if every school board member, superintendent, principal, parent, and legislative representative read the report as well.

The report is entitled, Searching for Computer Science: Access and Barriers in K-12 Education, and is available as a downloadable PDF here.

In short, the study illustrates that CS education matters, and that there is strong demand for CS education in K-12.  Two key barriers identified in the study are that administrators do not perceive that demand, and there is a shortage of teachers to teach CS.  The report also has some illuminating data around issues of equity and access.  As a whole, the report touches on critical topics we are working on in CSTA now and have even bigger plans for going forward.

Among some of the interesting findings:

  • 91% of parents surveyed want their child to learn Computer Science (CS), but <8% of administrators believe demand is high.
  • Half of principals and superintendents surveyed cite the lack of trained teachers as a barrier – greater than the need for technology. Less than half of administrators perceive school board support for CS.
  • 3 out of 4 principals surveyed say their school offers NO CS programming/coding classes.
  • More than 90% of students and parents surveyed believe people who do computer science have the opportunity to work on fun and exciting projects, and that people who do CS make things that help improve people’s lives.

Given that yesterday marked the “two-month mark” for me as the new executive director for the Computer Science Teacher’s Association, this news only reinforces for me that the mission of CSTA — to empower, support, and advocate for K-12 computer science teachers — is a mission that matters.

We can do more.  We must do more. CS matters, and we need more support to give all students better access to quality CS education, and ensure that CS teachers across the educational spectrum have access to the PD and resources they need to provide that quality CS education.  A great first step is making sure that the findings of this study are communicated and shared widely.

The Computer Scientist who Challenged the Nash Equilibrium

In November 2014, I was privileged to meet Constantinos Daskalakis, a Greek Computer Scientist and Associate Professor at MIT’s Electrical Engineering and Computer Science department. Daskalakis is best known for his work in the field of Game Theory, and his research on the Nash Equilibrium has earned him prestigious awards and an international reputation.

2014-11-18 11.54.32

Constantinos Daskalakis and Mina Theofilatou, November 2014

I had contacted Constantinos by e-mail earlier in 2014 regarding the awareness campaign Greek CS Teachers were running against the insane decision of the former Education Ministry’s administration to eliminate Computer Science from the National University Entry curriculum (read more about the problem here and about how it was resolved here) and, as one would expect, he was very supportive of our cause. When I wrote to him after booking my air travel for the CSTA November Board meeting – I had a 12-hour layover in Boston on the return leg, perfect for a visit to MIT! – he was willing to make some time in his hectic schedule to meet with me.

The meeting took place in his office at CSAIL in the Ray and Maria Stata Center; I had never been to Boston or MIT before so I followed the instructions he gave me (“it’s a crazy Frank Gehry building, you can’t miss it!,” he wrote). Our conversation naturally started with the situation in Greece, and inevitably ended with a discussion on the future for young computer scientists. I asked what he would advise exceptional students wishing to further their education at establishments such as MIT. He mentioned that what he is mostly looking for in Ph.D. applications is evidence of potential for ground-breaking research; that the clearest form of such evidence is prior research engagement; and that recommendation letters are crucial in assessing the student’s research potential.

That’s sound advice for those pursuing post-graduate studies, perhaps even an academic career. But what about Computer Science in Primary and Secondary Education? I asked Constantinos if he could write a few words about the importance of students having early exposure to computational thinking and computer science principles. Here is what he had to say:

“‘Information’ and ‘computation’ are just such fundamental concepts that there is no doubt they should be an integral part of primary and secondary education. All that takes place around us (or inside us) can be viewed as computing on information. Sometimes we want to process information efficiently—think Google trying to rank webpages. Other times we want to mine interesting information from vast amounts of raw information—think trying to identify chunks of the genome implicated in some disease. Sometimes we want to hide information—think cryptography. Other times we may want to release information while respecting privacy of individuals—think releasing medical data. Some other times, we want to incentivize individuals who have information to reveal that information to us…. And it is not just us trying to gain or operate on information. Biological, physical, and social processes are fundamentally computational, operating on information in some application-specific medium. Computer science principles can change one’s perspective on life, science, and society. Our educational system should teach that way of thinking early on.” 

And a summary of how his work has impacted international knowledge in computer science?

“My research studies the foundations of Economics from a computational standpoint. The starting point is that, besides sometimes being irrational, humans are definitely computationally bounded – in some sense, irrationality is a form of computational boundedness. So, studying economic behavior needs to incorporate computational thinking into economic thought. My research focuses at this interaction between Economics and Computation. One of my most celebrated results was showing that the Nash equilibrium – the crown jewel of prediction tools in Game Theory, defined by John Nash, the mathematician portrayed in “Beautiful Mind” – is computationally intractable. This means that, in complex interactions, it may take centuries of time before humans (even if they have access to super-computers to help them with their computation) behave as Nash equilibrium predicts. This casts doubt on how accurate the prediction ability of Nash equilibrium is, calling for better, computationally aware tools for predicting behavior. Research at the interface of Computation and Economics hopes to shed light on such issues of fundamental importance for understanding human behavior.”

A big thank you to Constantinos for providing valuable insight into the importance of Computer Science at all levels of education.

Mina Theofilatou, CSTA International Representative, Kefalonia, Greece

AccessCS10K: helping make high school computing courses more accessible and welcoming to students with disabilities


By: Richard E. Ladner
University of Washington

In 2010-2011, approximately 13 percent of K-12 students have a recognized disability according to the National Center for Education Statistics (NCES). This represents about 6.5 million students with disabilities in the K-12 education system. This only counts those students covered under the Individuals with Disabilities Education Act (IDEA) of 1990.   There are many more who have 504 plans (under the Rehabilitation Act of 1973). Students with 504 plans need some accommodation to attend school, but not an entire Individualized Education Plan (IEP) like those under IDEA. In any case, teachers of high school computing courses are likely to have students with disabilities in their classes.

One group of students, namely blind students, are already severely disadvantaged in high school computing classes that use programming tools such Scratch, Snap, Alice, Processing, Greenfoot, and other highly visual user interfaces that are generally inaccessible to blind users. Will teachers in these classes say, “Sorry you cannot take my computing course because you cannot use the tools we are using to teach programming?” Will a special education teacher or guidance counselor recommend to the student that she consider another career choice than the one the student wants to pursue? This should not happen. Computer science is a great profession for blind people and they should not be discouraged from pursuing it at an early age. (For profiles of blind individuals – and other individuals with disabilities – who have pursued computing visit the Choose Computing website.)

The goal of AccessCS10K is to help make it possible for all students with disabilities who are intellectually capable of learning computer science to have the opportunity to do so. The project is specifically focusing on the Exploring Computer Science (ECS) course and the Computer Science Principles (CSP) course, both of which are being supported by the National Science Foundation’s CS 10K initiative.

AccessCS10K has two objectives to reach its goal:

  • Build the capacity of ECS and CSP teachers to serve students with disabilities through professional development training and individualized real-time support.
  • Create accessible materials that ECS and CSP teachers can use in their classrooms, both tools and curricular units.

To build the capacity of teachers, AccessCS10K is partnering with as many of the CS 10K projects as possible to help them include information in their professional development about inclusive teaching strategies and accessible tools and curricula that can be used in the classroom. AccessCS10K holds capacity building institutes for the leaders of these projects as one way to help them integrate this information in their professional development. It maintains a community of practice whereby teachers can share resources among each other.   It maintains a searchable knowledgebase of articles that can help teachers find information they need to solve problems. Finally, it provides real-time support for teachers who have specific questions about how to integrate a student with a disability in their computing class. To get support call 509-328-9331 or e-mail

AccessCS10K is building accessible tools and curricula that can be used in ECS and CSP courses. One major tool is the Quorum language which is an easy to learn text-based programming language that can be used for both visual and non-visual projects. AccessCS10K’s development partners are working to make other accessible tools and curricula, not only for the programming components of these courses but for the other components as well.

In an upcoming blog post we will present an introduction to Quorum. If you are curious about Quorum we encourage you to visit the Quorum web site or to try the Quorum Hour of Code

AccessCS10K is funded by the National Science Foundation as part of the Computing Education for the 21st Century program of the Directorate for Computer & Information Science & Engineering (Grant #CNS1440843). AccessCS10k is a collaborative project between the University of Washington and the University of Nevada, Las Vegas.