Enabling children and beginning programmers to build AI programs

Click here if you wish to read this and the AI guide in a language other than English.

Introduction

The eCraft2Learn project is developing a set of extensions to the Snap! programming language to enable children (and non-expert programmers) to build AI programs. You can use all the AI blocks after importing this file into Snap! or Snap4Arduino. Or you can see examples of using these blocks inside this Snap! project.

AI programming guides

A guide that describes the extensions, projects, and the larger context currently consists of four chapters:

Adding speaking to programs Teacher guide Student guide
Adding listening to programs Teacher guide Student guide
Adding image recognition to programs Teacher guide Student guide
Adding machine learning to programs Teacher guide Student guide
Working with words and language New! Teacher guide Student guide

Publications

The paper AI Programming by Children by Ken Kahn and Niall Winters describes this work and will be published in the Proceedings of the Constructionism 2018 Conference in August. An early version of the AI extension to Snap! is described in the paper Child-friendly Programming Interfaces to AI Cloud Services also by Ken Kahn and Niall Winters.

Sample programs

The following demonstration programs use the AI Snap! blocks. They run best in the Chrome browser.

  1. Listen to generated speech with random pitch, rate, voice, and language  (requires speaker/headphones)
  2. Speak single word commands to a sprite (with synonym support)  (requires microphone and speaker/headphones)
  3. Speak full sentence commands to a sprite (with keyword search)  (requires microphone and speaker/headphones)
  4. Create funny sentences by verbally answering questions  (requires microphone and speaker/headphones)
  5. Customise stories by verbally answering questions  (requires microphone and speaker/headphones)
  6. Ask questions of Wikipedia  (requires microphone and speaker/headphones)
  7. Listen to a description of what is in front of the camera  (requires webcam and API key(s))
  8. Listen to a description of what is in front of the camera in response to you speaking  (requires microphone, speaker/headphones, webcam and API key(s))
  9. Train the turtle to move depending on which way your finger is pointed  (requires webcam and a GPU)
  10. Train the the turtle to move depending on the voice commands you give  (requires microphone)
  11. Train the turtle to move left or right depending on which way you lean  (requires webcam and a GPU)
  12. Play Rock Paper Scissors using machine learning  (requires webcam, speaker/headphones and a GPU)
  13. Play Simon Says using machine learning  (requires webcam, speaker/headphones and a GPU)
  14. Play Guess My Word that works by turning words into 300 numbers  (requires speaker/headphones and optional microphone)

We have collected some projects by students who have used these AI blocks.

Source code

The Snap! AI blocks load this JavaScript file on Github and then calls the functions defined therein.

Contact

Send email to Ken Kahn (toontalk@gmail.com) if you encounter problems or have questions.

GPU required for chapter 4 image recognition and some demos

Nearly all modern computers have a GPU. Without one or with a very primitive one programs using machine learning may run very slow. For example, the image machine learning commands run very slowly on Raspberry Pis. The audio training primitives, however, do run well on Raspberry Pis.

Using the AI blocks in your projects

You can import this file into your existing or new projects. The AI blocks can then be found in the Snap! palette under , , or .