Serial Communication

Table of Contents

1. L1: Intro to Serial
2. L2: Web Serial
3. L3: p5.js Serial
4. L4: p5.js Serial I/O
5. L5: Paint I/O Example
6. L6: ml5.js Serial
7. L7: HandPose Serial

---

In this lesson series, we will learn about Arduino-to-Computer interaction. More specifically, serial communication, web serial, and using p5.js to communicate bidirectionally with Arduino. By the end of this module, you’ll be able to build full end-to-end interactive systems that combine the power of web-based tools—including machine learning—with the physical interactivity of Arduino. 🔌

What you’ll learn in this module:

• How asynchronous serial communication works and how to use it for bidirectional Computer ↔ Arduino communication
• How to use the Web Serial API to build browser-based apps that talk to Arduino
• How to use p5.js for creative coding with serial input and output
• How to build complete interactive systems with bidirectional communication
• How to integrate machine learning (ml5.js) with Arduino for gesture- and pose-controlled physical computing

L1: Intro to Serial

In this lesson, we’ll dive into asynchronous serial communication and how we can use it for bidirectional Computer ↔ Arduino communication. We’ll show example serial communication clients using terminal programs and Python.

L2: Web Serial

In this lesson, you’ll learn about the Web Serial API and how to build simple web apps that communicate with Arduino directly from your browser.

L3: p5.js Serial

In this lesson, you’ll learn about Processing, p5.js, and how to use p5.js with Web Serial (with a focus on processing incoming serial input data).

L4: p5.js Serial I/O

In this lesson, we look at creating more complicated p5.js + Arduino applications that bidirectionally communicate together both Arduino to p5.js (Arduino → Computer) as well as p5.js to Arduino (Computer → Arduino).

L5: Paint I/O Example

In this lesson, we will bring everything together thus far to build a fully interactive p5.js + Arduino painting application, called PaintIO, with bidirectional serial communication and a custom paintbrush controller using an accelerometer to control brush location, a force-sensitive resistor to control brush size, and bimanual interaction using both Arduino + the mouse simultaneously for drawing.

L6: ml5.js Serial

In this lesson, we’ll introduce machine learning frameworks like ml5.js and show how to use them with Arduino. Specifically, we’ll use ml5’s BodyPose model for real-time pose estimation and send body keypoint data to Arduino via Web Serial.

L7: HandPose Serial

In this lesson, we’ll explore ml5’s HandPose model for real-time hand tracking and use it to control a servo motor on Arduino. We’ll also build a fun physical construction (a cardboard figure that waves back at you!) to demonstrate the connection between ML-based input and physical output.