Enabling children and beginning programmers to build AI programs

Click here if you wish to read this and the AI guide in a language other than English.

Introduction

The eCraft2Learn project is developing a set of extensions to the Snap! programming language to enable children (and non-expert programmers) to build AI programs. You can use all the AI blocks after importing this file into Snap! or Snap4Arduino. Or you can see examples of using these blocks inside this Snap! project.

AI programming guides

A guide that describes the new blocks, possible projects, sample programs, background, and the larger context currently consists of six chapters:

Adding speaking to your programs Speech synthesis requires only speakers or earphones
Adding listening to your programs Speech recognition requires a microphone (builtin or connected to a USB port)
Adding image recognition to your programs Requires a camera (builtin or connected to a USB port) and registering to get API keys
Adding machine learning to your programs Many examples rely upon a camera. It is very slow unless your device has a GPU.
Working with words and language No special hardware requirements
Making machine learning neural nets New! It is very slow unless your device has a GPU.

Publications

The paper AI Programming by Children by Ken Kahn and Niall Winters describes this work and was published in the Proceedings of the Constructionism 2018 Conference. AI Programming by Children using Snap! Block Programming in a Developing Country by Ken Kahn, Rani Megasari, Erna Piantari and Enjun Junaeti was published at the EC-TEL 2018 conference and describes workshops with high school students in Indonesia using these AI blocks. An early version of the AI extension to Snap! is described in the paper Child-friendly Programming Interfaces to AI Cloud Services also by Ken Kahn and Niall Winters.

Presentations

This work has been presented at several conferences and universities. The slides can be found here.

Sample programs

The following demonstration programs use the AI Snap! blocks. They run best in the Chrome browser.

Listen to generated speech with random pitch, rate, voice, and language
(requires speaker/headphones)
Speak single word commands to a sprite (with synonym support)
(requires microphone and speaker/headphones)
Speak full sentence commands to a sprite (with keyword search)
(requires microphone and speaker/headphones)
Create funny sentences by verbally answering questions
(requires microphone and speaker/headphones)
Customise stories by verbally answering questions
(requires microphone and speaker/headphones)
Ask questions of Wikipedia
(requires microphone and speaker/headphones)
Ask about the weather anywhere in the world
(requires microphone and speaker/headphones)
Listen to a description of what is in front of the camera
 (speaker/headphones, webcam, and API key(s))
Listen to a description of what is in front of the camera in response to you speaking
(requires microphone, speaker/headphones, webcam, and API key(s))
Train the turtle to move depending on which way your finger is pointed
(requires webcam and a GPU)
Train the the turtle to move depending on the voice commands you give
(requires microphone)
Train the turtle to move left or right depending on which way you lean
(requires webcam and a GPU)
Play Rock Paper Scissors using machine learning
(requires webcam, speaker/headphones and a GPU)
Play Simon Says using machine learning
(requires webcam, speaker/headphones and a GPU)
Play Guess My Word that works by turning words into 300 numbers
(requires speaker/headphones and optional microphone)
Turn a photo into 8 different paintings
(requires webcam (but easily modified to use any image instead) and a GPU)
Translation using 300 numbers and a word list with 2.5% coverage
(no special requirements)
Guesses the relationship between two lists of numbers
(runs very slowly without a GPU)
Learns to play Tic Tac Toe
(runs very slowly without a GPU)
Tries to solve analogy problems with photos (partially successfully - can you improve it?)
(runs very slowly without a GPU)

We have collected some projects by students who have used these AI blocks.

Source code

The Snap! AI blocks load this JavaScript file on Github and then calls the functions defined therein.

Contact

Send email to Ken Kahn (toontalk@gmail.com) if you encounter problems or have questions.

GPU required for chapter 4 image recognition and some demos

Nearly all modern computers have a GPU. Without one or with a very primitive one programs using machine learning may run very slowly. For example, the image machine learning commands run very slowly on Raspberry Pis. One version of the audio training primitives, however, do run well on Raspberry Pis.

Using the AI blocks in your projects

You can import this file into your existing or new projects. The AI blocks can then be found in the Snap! palette under , , or .