F15: Electronic Piano

Motion Controlled Electronic Piano

Top view of implemented board

Piano keys representation of 1 key short of an octave

Side view of implemented board

Top keys being played

Bottom keys being played

Abstract

The purpose of this project is to design and build a motion controlled electronic piano powered by an SJ-One Board. Users will be able to play the notes of a piano using IR proximity sensors as keys. At activation, the key's LED will light up. The number of keys of the piano will cover one key short of an octave; however, the piano provides users the capability to change (offset) the notes across other ranges.

Objectives & Introduction

The objective of this project is to design and build a multi-keyed motion controlled electronic piano. The piano will feature 12 IR proximity sensors to act as motion activated keys, a real-time MIDI decoder to play piano notes, and LEDs that activate along with the activation of its associated key. The electronic piano will utilize FreeRTOS multitasking environment to simultaneously sample values from multiple IR proximity sensors, analyze the obtained values, then play piano notes according to the active state of the IR keys.

1. Read an IR sensor's value using an external ADC

2. Write a software to read multiple IR sensors and play different piano notes using a MIDI decoder

3. Assemble a range of IR sensors to function as piano keys and an array of ADCs to read all IR sensors

4. Build a wooden frame and mount the control circuits and IR sensors

5. Integrate LEDs to each key and control the LEDs using an LED driver

Team Members & Responsibilities

Jason Tran
- Design and write the software
- Design and build the power circuit and ADC circuit
Arthur Nguyen
- Design and build the wooden frame
- Mount boards and IR keys onto the wooden frame
- Design and build the LED circuit

Schedule

Week#	Date	Task	Actual
1	10/24-10/30	Project proposal	Completed. Discussed project design of both hardware and software.
2	10/31-11/6	Part testing and project outline	Completed testing Sharp IR sensor and MIDI controller interface.
3	11/7-11/13	Order parts	Completed order of essential parts including VS1053, short IR sensors,IR jumper wires, and analog to digital converters. Completed software.
4	11/14-11/20	Setup all keys with sound.	Completed playing sounds with key activation.
5	11/21-11/27	Volume control and power circuit.	Completed.
6	11/28-12/4	Combine circuits and start mounting system.	Ordered LEDs and LED driver. Started mounting system onto board.
7	12/5-12/11	Build LED circuit and implement software for LEDs. Combine circuits and mount rest of system onto board.	Completed mounting circuit boards and wiring on 12/15. Final test completed on 12/16.
8	12/12-12/17	Final changes and demo	Completed demo on 12/17.

Parts List & Cost

Quanitity	Price per part	Part	Detail	Vendor
1	$80	SJ-One Board	Microcontroller	SJSU SCE
12	$13.95	GP2Y0A41SK0F	Infrared Proximity Sensor Short Range-Sharp	SparkFun
12	$1.50	3-pin JST	Infrared Sensor Jumper Wire	SparkFun
6	$2.30	MCP3002	Analog to Digital Converter	SparkFun
1	$19.95	VS1033D	MIDI Decoder Breakout Board	SparkFun
1	$1.95	LD1117V33	3.3 V voltage Regulator	SparkFun
1	$1.50	PRT-08032	3.5 mm audio jack	SparkFun
2	$1.25	PRT-09011	USB female type A speakers	SparkFun
1	$13.00	AL-101	USB powered speakers	Amazon
24	$0.04		150 Ohm resistor	Fry's
12	$0.202	WP59EGW/CA	Red/Green Diffused Common Anode LED	Mouser
1	$1.48	TLC6C5912QPWRQ1	12 Channel Shift Register LED Driver	Mouser
1	$5.42	LCQT-TSSOP20	TSSOP to 20DIP Socket Adapter	Mouser
Total	$328.38

Design & Implementation

Hardware Design

   Board Schematic

Top view schematic

The board schematic shown displays the precise measurement of the board as well as where the keys are placed on the board. Measurements were made with hand size taken into account such that the average hand, when centered on one key, will not overlap onto nearby keys and accidentally activate them. Keys are also assigned 0 to 11. This assignment is referenced by the LED numbering and the ADC_Ch signal. LEDs are placed beneath the keys, with red LEDs associated to the top keys and green LEDs associated to the bottom keys. The difference between bottom and top keys is about 5 cm in length, which was created in order to differentiate between choice of key being played between top and bottom keys.

    Power Circuit

Power Circuit schematic

Power Circuit

The power circuit can be powered by a USB power supply which distributes the source to the SJ-One Board, to the 3V3 voltage regulator, and to another USB A connector intended to supply power to an external speaker. The 3V3 voltage regulator distributes power to the VS1033D MIDI decoder, to the ADC circuit, and to the LED driver. The power circuit also contains a 3.5mm audio out connector to direct the audio signal from the MIDI decoder to an external speaker; right and left channel are tied together at the connector. Additionally, the board contains an AC coupling circuit to remove the DC offset from the audio signal.

    SJ-One Board

SJ-One Board

The SJ-One Board is powered by the 5V and ground supplied by the power circuit. The CLK pin is outputted to the VS1033D, analog to digital converters, and LED driver. The MOSI pin is outputted to the VS1033D, analog to digital converters, and LED driver. The MISO pin is only outputted to the analog to digital converters. There are also 8 GPIO output pins used for chip selecting. P1.30 is used for selecting the VS1033D, while P2.4 is used for selecting the LED driver. Pins P1.29, P1.28, P1.23, P1.22, P1.20, and P1.19 are used for selecting between analog digital converters 0 to 5, respectively. Four GPIO inputs are used for user control. Pins P1.14 and P1.15 are used for lowering and raising the volume, respectively. Pins P1.9 and P1.10 are used for lowering and raising the note selection, respectively. LEDs 0 to 3, which are controlled by GPIO output pins P1.0, P1.1, P1.4, and P1.8, respectively, are used to display current volume, while the 7-segment LEDs represent the note selection.

    VS1033D Breakout Board

VS1033D Breakout Board Schematic

VS1033D Breakout Board

The VS1033D functions as a real-time MIDI decoder; piano notes can be played by sending MIDI standard messages via SPI to the device; audio is obtained from analog output RIGHT. For each MIDI data frame, a byte 0xFF must be sent after. To enable RT MIDI mode, IO1 must be HIGH and IO0 must be LOW on power up. Externally undriven input pins not shown in the schematics, like IO0, have pull-down resistors within the PCB. The chip select CS (active low) is not used in RT MIDI mode; instead, BSYNC functions as chip select. The VS1033D UART controller is not used; Rx must be tied HIGH to remain inactive. Its Serial Out (SO) is left unconnected (labeled as NO_CONNECT), since its returning data is not used.

    Analog to Digital Converter Circuit

Single analog to digital converter

ADC Circuit schematic

ADC Circuit

As seen, the MOSI, MISO, and CLK connections from the SJ-One Board connect to all of the analog to digital converter's Din, Dout, and CLK pins, respectively. 6 GPIO pins from the SJ-One Board are tied to each of the 6 CS pins of the analog to digital converters. VDD is connected to a 3.3V source. Finally, the channels of the ADC are connected to the Vout of the short IR sensors.

    IR Proximity Sensor

IR Proximity Sensor

The IR proximity sensor functions as a short-ranged distance sensor. The electronic piano consists of 12 of these sensors; each of the sensor's analog voltage output is connected to a unique ADC channel ADC_CH11 through ADC_CH0 of the Analog to Digital Converter Circuit.

    LED Driver and LEDs

LED driver circuit

Top key LED

Bottom key LED

As seen, Drain0 to Drain11 are connected to 150 Ohm resistors, which in turn, are connected to their corresponding numbered key LED cathodes. All LEDs have their anodes connected to a 5 V source. LEDs 0, 1, 2, 3, 4, 5, and 7 have its green cathode (shorter) connect to 150 Ohm resistors and then to ground in order to illuminate green color all the time. Likewise, LEDs 10, 11, 8, 9, and 6 have their red cathodes (longer) connected to 150 Ohm resistors and then to ground in order to illuminate red color at all times. The cathodes are connected to 150 Ohm resistors such that the current going through the LED is about 18 mA for green and 20 mA for red. Drains 0, 1, 2, 3, 4, 5, and 7 are connected to 150 Ohm resistors and then to their corresponding LEDs' red cathode. Drains 10, 11, 8, 9, and 6 are connected to 150 Ohm resistors and then to their corresponding LEDs' green cathode. The GND and G pins are connected to ground, while the VCC and CLR pins are connected to 3.3 V source. The MOSI, CLK, and GPIO output pins from the SJ-One board are connected to the SER IN, SCK, and RCK pins of the LED driver.

Hardware Interface

SPI was used to communicate with the 6 ADC devices, the MIDI decoder, and the LED driver. Meanwhile, GPIO was used as control signals, such as chip select and output enable, for the MIDI decoder and the LED driver. In the software, the following SPI and GPIO driver were used to communicate to the IO devices.

SPI driver:

   namespace My_spi {
       void init_ssp1_spi( void );                   // Using SCK1, MISO1, MOSI1 (P0.7, P0.8, P0.9)
       void init_cs( uint8_t port, uint8_t pin );
       char spi_exchange_byte( char byte );
       void cs_select( uint8_t port, uint8_t pin );
       void cs_deselect( uint8_t port, uint8_t pin );
   }

The init_ssp1_spi() function initializes LPC1758 SSP1 controller to function as an SPI controller with the following setting: MSB first, 8-bit data frame, and 1 MHz clock while other settings remain as default values.

The init_cs(port, pin) function initializes a given IO pin to function as a GPIO output used for chip select. cs_select(port, pin) and cs_deselect(port, pin) functions are used to activate and deactivate a given IO pin respectively.

The spi_exchange_byte(byte) function clocks 8-bits to and from a SPI slave device; cs_select(port, pin) and cs_deselect(port, pin) should be called before and after the call of the SPI exchange function(s) to target a specific SPI slave device. Exception: The LED driver is a shift register and does not use chip select. Instead, a GPIO output is used to signal the shift register to update its output (LEDs).

GPIO driver:

   namespace My_gpio {
       void set_gpio_mode( uint8_t port, uint8_t pin, bool mode );
       void digital_write( uint8_t port, uint8_t pin, bool state );
       bool digital_read( uint8_t port, uint8_t pin );                     // Assuming active high input
   }

The set_gpio_mode(port, pin, mode) function initializes a given IO port to function as a specific GPIO mode (mode can be either input or output).

The digital_write(port, pin, state) function sets a given GPIO pin to a given voltage level or state (voltage level can either be high or low).

The digital_read(port, pin) function reads the voltage level of a given pin and returns a Boolean value (returns true if high and false if low).

Software Design

For the software design, there are three tasks: MIDI task (consumer), IR Key task (producer) and Control task (producer). During the idle state, the consumer remains in sleep and waits for a command from the producers. When an action takes place such as an activation of a key or a button, the consumer immediately wakes and executes the received command. Commands and messages are passed using FreeRTOS queues.

The MIDI task is responsible for playing notes at a certain pitch and volume. The task also provides feedback of the current pitch (octave) and volume using the SJ-One Board’s 7 segment display and on-board LEDs.

The IR Key task is responsible for sampling the IR sensors by reading the values from the ADC. If a key is sampled as active, then the key’s LED activates and the key’s note in addition to the note on command are sent to the MIDI task. The opposite occurs when a key is sampled inactive when previously in the active state such that the key's LED deactivates and the key's note is sent to the MIDI task to be deactivated.

The Control task is responsible for reading user input to increment/decrement pitch or volume by sampling the SJ-One Board’s on-board push button. When an input is active, the appropriate command gets sent to the MIDI task.

Implementation

MIDI Task Algorithm

1. Sleep and wait for a command.

2. If the play note command is received, then send the MIDI standard message via SPI to the MIDI decoder to play each note in the Note Queue.

3. If the increment/decrement volume command is received, then send the MIDI standard message via SPI to the MIDI decoder to increment/decrement the volume of the MIDI decoder.

4. If the increment/decrement pitch command is received, then offset base note by one range.

5. Return to sleep and wait for another command (repeat from step 1).

IR Key Sampling and Activation Algorithm

1. Using SPI, read 12 ADC channels (6 ADC, 2 channels each) to obtain the values of all 12 IR sensors.

2. Compare each value of the 12 IR sensors against valid margin levels.

3. If a key meets the valid active margin level, the note of the key is sent to the MIDI task Note’s Queue to be played. (To compensate for noise, a key must be sampled as valid 3 times before being queued to be played.)

4. For keys previously in the active state, if its value meets the valid inactive margin level, then the note of the key is sent to the MIDI task Note’s Queue to be turned off.

5. After all 12 values are checked, if any note had been sent to the MIDI task’s Note Queue, then update the key’s LED based on its new state.

6. Then, after all 12 values are checked AND if any note had been sent to the MIDI task’s Note Queue, send the command Play Note to the MIDI task’s Command Queue.

7. Reread ADCs (repeat from step 1).

Control Task User Input Sampling and Activation Algorithm

1. Read the state of the SJ-One Board’s on-board push button.

2. If the increment pitch button is sampled active, then send the increment pitch command to the MIDI task; similarly, if the decrement pitch button is sampled active, then send the decrement pitch command to the MIDI task.

3. If the increment volume button is sampled active, then send the increment volume command to the MIDI task; similarly, if the decrement volume button is sampled active, then send the decrement volume command to the MIDI task.

4. Reread states of on-board push buttons (repeat from step 1).

Testing & Technical Challenges

Testing was done on individual parts and then on combined parts as part of a circuit. Finally, testing was done on the project as a whole. First, testing was done on the IR sensors in order to read values while moving our hands over it and using the SJ-One Board's analog to digital converter. At the same time, testing was done on the VS1033D by sending MIDI messages to it and playing piano notes through headphones. Next, the IR sensors were attached to the VS1033D in order to activate notes when activated by motion. The 6 external analog to digital converters are then tested for reading input via motion from 12 IR sensors simultaneously. This circuit was then attached to the VS1033D in order to test for activation of different notes with its associated key. Finally, the LED driver was tested separately by sending SPI messages in order to activate specific LEDs. Finally, the LED driver was integrated with the rest of the system; an activation of a key should activate its specific note and LED.

Issue #1-ADC chip select

A problem occurred when testing the ADC circuit. During the test, one ADC was tested; the unused ADCs' chip selects remained floating; this resulted in bus contention. The problem was fixed by driving all other unused CS to HIGH (inactive), while the CS of the ADC under test was driven LOW (active).

Issue #2-LED driver soldering

At first, there was difficulty implementing the LED driver circuit. After soldering the LED driver (shift register) to the TSSOP to DIP20 adapter, a short may had existed and the device was not functioning. After improving the soldering, the device functioned properly; the expected LEDs were turned on based on the data sent to it.

Issue #3-Noisy power supply

Powering 12 IR proximity sensors introduced noise in the 5V supply line; for this reason, there are three major problems.

For the first problem, the voltage of the 5V supply drops to approximately 4.5V-4.2V; supplying inadequate voltage to the system can lead to unpredictable results. To mitigate the problem, a higher-quality power supply was used to reduce the noise and voltage drop. Additionally, a 470uF bypass capacitor was added to the power circuit to improve the noise filtration.

For the second problem, false key activation can occur; infrequently, random keys would unintentionally activate without any action from the user. To mitigate the problem, IR debouncing was implemented in the software. Similar to push button debouncing, an IR key must be sampled valid active multiple times in order to play its note.

For the third problem, an external speaker that uses the 5V USB connector of the power circuit as a source would amplify the noise and idly emit high pitch static. To mitigate the problem, an external speaker with its own power supply was used to partially isolate the noise in the supply line from the audio. Using this approach (although not an ideal solution), the high pitch static was significantly reduced. Since the source of the noise is from the array of IR sensors, a cost effective solution is to use a passive filter to capture and isolate the noise between the power circuit and the ADC circuit.

Noise in the 5V supply line

Conclusion

This project was completed successfully. The project began with a simple idea of creating a motion controlled electronic piano. Next, we thought of how this would be built and what parts were needed in order to complete this task. We were able to theoretically build the entire system and determine how it functioned through FreeRTOS queues. Once parts were determined, testing was done for each individual part in order to verify its function. Once parts were successfully tested, they would be combine in order to create a bigger system, and eventually, the entire piano system. From the project, we were able to learn how to implement the LPC1758's serial communications in order to build an embedded system. The implementation involved both hardware and software capabilities, which ultimately led to the creation of the electronic piano.