Peter at the age of 6 is fascinated by what his dad was doing with the computer and the various toy robots in his workshop. His dad is a software engineer, and his job is to make "robots that can talk".

One day, Peter decided to tell his dad, that he wanted to learn how to make robots that can talk. His father looked at his determined face and agreed to his request.

Through Peter's learning journey, you will understand some considerations required to begin exploring code projects such as:

• Identify the question and the end-goal
• Design for the solution in a backwards flow
• Overcome learning barriers by learning how to learn
• Understand the process in reading and running code projects

Are you ready to begin exploring with Peter?

Here is the list of resources and generic reference points compiled for Peter's Journey:

• [Resource] Link to Zip file (containing the 3 code tutorials and accompanying test data): https://drive.google.com/drive/folders/1nUxQG232eXBShz2qWJk24RKhjtFJ0aUE?usp=sharing
• [Generic Reference] Python: https://www.python.org/
• [Generic Reference] Google Colab: https://colab.research.google.com/

Peter's perspective: So you want to create a robot that can talk...

"What is the first thing you should do?" Peter's dad asked.

"Waiting for you to teach me." Peter replied with a frown.

"Fair enough, but why do you want to create a robot that can talk?" Peter's dad asked.

"For... fun...?" Peter answered. "And for it to tell me stories since you and mom are too busy."

"Ah... I see. And how much time will you want to spend on this "fun" project." Peter's dad asked further.

"I dunno, how about a week?" Peter ask.

"Do you want to build the body of the robot?" Peter's dad probe further.

"It looks difficult, I just want a robot that can talk." Peter said.

Okay, now that I have understand your objectives, let me create the steps for you to train a robot that can talk to you."

Peter's Learning Journey...

What are the ingredients making up the machine brain for a robot that can talk?

A machine brain that allows a robot to "talk" and conduct certain "actions" will require "capabilities" and "skills" that have to be trained.

• The machine brain needs to be able to recognise voice commands. This can be done by converting voice to text. The human can then associate certain text to "dialogue flows" and "actions" to be delivered or completed by the robot based on pre-set and pre-trained instructions.
• When a robot is switched on, the machine brain is listening in the background for keywords to "wake up".
• When "wake up" keywords are triggered, the machine brain can then tap on its processing powers to convert audio to text.
• Converted text are then fed into the brain who will produce a dialogue or action based on pre-trained instructions from a human.
• Data are converted to audio/mechanical action for the robot to provide a verbal/physical response to the human's voice commands which triggered the task flow.

Peter's Perspective: Overcoming the barriers to learning

"I don't think I can build a machine brain." Peter said discouraged after the explanation. "It is too difficult.'

"You can if I help you." Peter's dad replied before continuing. "How did you learn how to ride a bicycle?"

"You teach me." Peter noted. "And walk with me until I can ride in a straight line."

"It is the same thing when using code to setup a machine brain. We do it in pieces, in a linear (straight line) sequence first before we build an iterative (repetitive) sequence to improve the brain. And we are not doing it alone. A lot of people have tried to solve one piece of the code and shared it. So there is a shared pool of 'lego bricks of code in the internet." Peter's dad answered.

"I will be sharing my own code bricks with you. All you have to do is to read and amend some parts of the code, so that you can build up a connected flow of code pieces that can do what you want."

"Okay... I guess I can try. So how do I read and write code?" Peter asked.

"To do that you have to pick a programming language that can help you 'talk' to the computer." Peter's dad explained. "Let's choose Python."

Selecting a programming language

At this moment in time (in the 21st century), there are around 50 popular programming languages although there is an estimate of around 700 languages from past to present.

Some programming languages are high-level, designed for human usability but covers less detail and is limited in the instructions it can pass to the machine.

Some programming languages are low-level, designed for deeper engagement with the machine, structurally similar to a machine processor’s instructional flow, but are harder to use by a human.

A computer programmer might be from domains outside of computer science or engineering. There are also a number of hobbyist programmers who could have picked up a programming language just to develop their own software ideas or play with robotics.

As a rule of thumb when picking-up a programming language, for your primary programming language, choose one that is:

• in use by majority,
• in use by majority,
• is able to translate to the machine your intended programming activities,
• has strong community support,
• active updates on the language packages, libraries, tutorials etc.
• and in demand by the wider industry.

Depending on the time available to you and your passion in creating new digital products, you may decide to pick up a secondary programming language to cover the gaps inherent in your primary programming language, or to learn for fun.

For your learning journey, let's start by learning a bit of Python programming languages.

Self-Assessment

How many computer programming languages do you know and can apply so far?

Peter's Perspective: Learning how to learn

"What does mom's workspace looks like?" Peter's dad asked.

"A lot of magazines and picture books. Her laptop, iPad, stylus and drawing pad. She also have a lot of drawing boards and paint brushes." Peter mused. "And her scrapbooks."

"That's because she is an artist and those are her personal preferred tools of her trade." Peter's dad explained. "She uses magazines and picture books to learn and get inspirations, cutting out the important pieces to save in her scrapbooks, as easy to reference libraries of ideas. She practices with paper, pen and paintbrushes, and do digital artworks using her laptop which is installed with the software that she needs. Now let's talk about you. What do you think you will need to create a machine brain?"

"You said I need to learn Python to talk to the machine, do I also need a computer?" Peter asked.

"Yes! You will need a computer and keyboard to type your code. You will also need a software environment that allows the code to be translated into instructional packages, that the machine can take on to execute. The environment should optimally be able to allow you to test your code besides running it." Peter's dad explained.

"As you are just trying out, the system or process for you to create your digital project could be structured this way:

• Start a digital folder to keep your code references, half completed, and completed projects.
• Open a notepad or document file to note down the steps you take to create the project, the problems encountered, and how it is resolved.

These references and notes will form your learning and reference libraries, to help you do things faster and smoother when you try out similar projects in future."

"For the developer's environment, we will be using Google Colab which can be run on the internet (cloud). As a start, you will just need to run through the code blocks to get a feel of the environment, the process of how such code projects can be arranged and completed in sequential flow. Perhaps through this exercise, you can then be aware of what learning a programming language can actually do for you."

Setup your coding environment and resources in Google Colab

• First, using your Google account (create a free one if you have yet to register for one).
• Go to https://research.google.com/colaboratory to associate your Google account with the Google colab feature.
• Download the zip file (0_MOOC_Peter's Journey.zip) from this Google Drive link.
• Extract the 0_MOOC folder from the zip file, there should be 6 resources contained within:
  • 3 Jupyter Notebook
    • 1_1_Tutorial_MicRecording Via Browser.ipynb
    • 1_2_Speech to Text.ipynb
    • 1-3_Tutorial_Text to Speech.ipynb
  • Text files
    • story_1.txt
    • story_2.txt
    • story_3.txt
• In your Google Drive > Enter your Colab Notebooks folder > Drag and Drop the 0_MOOC folder and its 6 resources into the folder.

Hands-on Warm-up: Hello World

Within your Google Drive and Colab Notebooks folder, enter your 0_MOOC folder.

• First let's do some warm up to get yourself familar with Google's Colab function.
• At the top left corner of the browser, under the Google Drive icon, click on New and Select More. There will be a panel pop up with more options.
• Select Google Colaboratory.
• Give your browser sometime to communicate with the Google server. You should see a new browser tab appearing as a Jupyter Notebook is opened.
• Rename the Jupyter Notebook to 1_0_Warm Up.
• If this is your first time using a Jupyter Notebook, Jupyter Notebook is actually an open document format based on JSON. The tool can help record, frame and execute Python code. With Google Colab, you can create a web version that run out of your Google Drive.
• Take note that the cells comes in two types. Text and Code. For notation and code documentation, it is advisable to leave a descriptive text cell above a critical code / project feature milestone.
• For this Warm up exercise, create a text cell and type: We will be getting our machine to respond to our instruction, by getting it to print the following message "Hello World".
• Run each code cell/block by clicking on the "play" icon at the left hand side of each code block. The (web) computer system will process the code, once it has completed its task, a green tick will show below the code cell.
• Should you wish to leave a description in the code cell, type "# <description>" e.g., "# the following code will do the..."

Tutorial 1: Activating microphone and recording voice through browser

For a robot or smart assistant with voice command and reactive features, one of the potential design feature is to allow the machine to "listen" in the background and only react when certain "command words" are spoken. This might tie-in with some energy savings mechanics or to constraint the reactiveness of the machine in a specific "activation" period.

For Peter's pet project, as there is no need for such complexity, let's start with just the bare minimum foundation instead.

What is the first step in this case?
Ans: As a first step, we need to instruct the machine to capture the human voice and save it into a digital audio format that the machine can utilise for future actions.

You can open 1_1_Tutorial_MicRecording Via Browser.ipynb to get started in running and testing out the code blocks.

You may follow the video demo to understand the flow of work.

If you have not yet download the zip files to access the source codes, you may click on this Google Drive link. To setup the resources in your Google Drive so that you can follow the tutorials, refer to:

• Setup Your Coding Environment And Resources

Tutorial 2: Convert audio clip to text using speech to text technology

Speech-to-text software at this moment in time are primarily built with machine learning and deep learning technology. Building such software from scratch will take a lot of manual effect, data collection tasks and development time. As such we will be tapping on the free access version of Google's Speech-to-Text engine which can translate audio file of limited duration and size.

Now that we have the audio file of the human speech command saved by the computer system, what is the next step?
Ans: With the audio file, the next step is to convert the audio file into text format. This is to assist the human programmer in checking the accuracy of the machine's hearing, and to associate the text commands with decision-making and reply system at the later stage.

You can open 1_2_Speech to Text.ipynb to get started in running and testing out the code blocks.

You may follow the video demo to understand the flow of work.

If you have not yet download the zip files to access the source codes, you may click on this Google Drive link. To setup the resources in your Google Drive so that you can follow the tutorials, refer to:

• Setup Your Coding Environment And Resources

Tutorial 3: Setting-up a rule-based decision-making and voice reply system

Similarly with text-to-speech software at this moment in time building such software from scratch will require significant investment of time and effort. As such we will be tapping on the free access version of Google's Text-to-Speech engine which can translate text to audio output of limited duration and size.

Now that we have the textual understanding of the human speech that was recorded, what is the next step?
Ans: For simple decision-making, we can setup a rule-based system to check for certain text inputs and cater the machine's reply in a certain way. In the case of Peter's pet project, we can design for "stories" to be read as external files which can be loaded into the machine's instructional flow if the user's command is for it to "tell a story".

You can open 1_3_Tutorial_Text to Speech.ipynb to get started in running and testing out the code blocks.

You may follow the video demo to understand the flow of work.

If you have not yet download the zip files to access the source codes, you may click on this Google Drive link. To setup the resources in your Google Drive so that you can follow the tutorials, refer to:

• Setup Your Coding Environment And Resources

Ideation Exercise

With the 3 Jupyter Notebooks shared, how will you use the code blocks available to create a digital product?

Write, sketch out your idea or just record your thoughts in a journal.

Did you know that there are programmes in Singapore to help kids learn coding?

Google and IMDA has a Code in the Community (CITC) programme. This is Singapore’s largest program providing free coding classes to get children from disadvantaged backgrounds interested in coding.

• Ref: https://codeinthecommunity.com/
• Ref: https://www.imda.gov.sg/news-and-events/Media-Room/Media-Releases/2020/Google-and-IMDA-expand-support-for-Code-in-the-Community

Digital for Life brings together Singaporeans from all ages and walks of life to embrace digital learning as a lifelong pursuit.

• Ref: https://www.digitalforlife.gov.sg/

AI Singapore has a mandate to grow local (Singapore) expertise in AI and to help move industry entities out of AI Unaware stage to basic awareness and expertise stages, aligning with their AI Readiness framework. They also have a programme targeted at kids known as AI for Kids (AI4K).

• Ref: https://learn.aisingapore.org/students-2/
• Ref: https://aisingapore.org/innovation/airi/

Institute of Technical Education (ITE) has skills future courses in AI and coding topics at an applied and beginner level.

• Ref: https://www.ite.edu.sg/courses/part-time-courses/skillsfuture-series-certificate-of-competency/electronics-infocomm-technology