Difference between revisions of "F15: Electronic Piano"

From Embedded Systems Learning Academy
Jump to: navigation, search
(Project Title)
(Grading Criteria)
 
(308 intermediate revisions by one other user not shown)
Line 1: Line 1:
=== Grading Criteria ===
+
== Motion Controlled Electronic Piano ==
<font color="green">
+
[[File:CMPE146_F15_ElectricPiano_TopView.jpg|framed|boarder|left|text-top|Top view of implemented board]]
*  How well is Software & Hardware Design described?
+
[[File:CMPE146_F15_ElectricPiano_PianoKeys.jpg|framed|boarder|center|text-top|Piano keys representation of 1 key short of an octave]]
*  How well can this report be used to reproduce this project?
+
[[File:CMPE146_F15_ElectricPiano_SideView.jpg|framed|boarder|left|text-top|Side view of implemented board]]
*  Code Quality
+
[[File:CMPE146_F15_ElectricPiano_TopPlay.jpg|framed|boarder|center|text-top|Top keys being played]]
*  Overall Report Quality:
+
[[File:CMPE146_F15_ElectricPiano_BottomPlay.jpg|framed|boarder|center|text-top|Bottom keys being played]]
**  Software Block Diagrams
+
 
**  Hardware Block Diagrams
+
 
**: Schematic Quality
 
**  Quality of technical challenges and solutions adopted.
 
</font>
 
  
== Project Title ==
 
Electronic Piano: Motion Controlled LED Keys
 
  
 
== Abstract ==
 
== Abstract ==
This section should be a couple lines to describe what your project does.
+
The purpose of this project is to design and build a motion controlled electronic piano powered by an SJ-One Board. Users will be able to play the notes of a piano using IR proximity sensors as keys. At activation, the key's LED will light up. The number of keys of the piano will cover one key short of an octave; however, the piano provides users the capability to change (offset) the notes across other ranges.
  
 
== Objectives & Introduction ==
 
== Objectives & Introduction ==
Show list of your objectives. This section includes the high level details of your project. You can write about the various sensors or peripherals you used to get your project completed.
+
 
 +
The objective of this project is to design and build a multi-keyed motion controlled electronic piano. The piano will feature 12 IR proximity sensors to act as motion activated keys, a real-time MIDI decoder to play piano notes, and LEDs that activate along with the activation of its associated key. The electronic piano will utilize FreeRTOS multitasking environment to simultaneously sample values from multiple IR proximity sensors, analyze the obtained values, then play piano notes according to the active state of the IR keys.
 +
 
 +
1. Read an IR sensor's value using an external ADC
 +
 
 +
2. Write a software to read multiple IR sensors and play different piano notes using a MIDI decoder
 +
 
 +
3. Assemble a range of IR sensors to function as piano keys and an array of ADCs to read all IR sensors
 +
 
 +
4. Build a wooden frame and mount the control circuits and IR sensors
 +
 
 +
5. Integrate LEDs to each key and control the LEDs using an LED driver
  
 
=== Team Members & Responsibilities ===
 
=== Team Members & Responsibilities ===
Yasmine Jamjoum
+
Jason Tran
**  
+
** Design and write the software
Khang Doan
+
** Design and build the power circuit and ADC circuit  
**
+
*  Arthur Nguyen
 +
** Design and build the wooden frame
 +
** Mount boards and IR keys onto the wooden frame
 +
** Design and build the LED circuit
  
 
== Schedule ==
 
== Schedule ==
Show a simple table or figures that show your scheduled as planned before you started working on the project.  Then in another table column, write down the actual schedule so that readers can see the planned vs. actual goals.  The point of the schedule is for readers to assess how to pace themselves if they are doing a similar project.
 
  
 
{| class="wikitable"
 
{| class="wikitable"
Line 37: Line 45:
 
|-
 
|-
 
! scope="row"| 1
 
! scope="row"| 1
| 10/8
+
| 10/24-10/30
| Task list
+
| Project proposal
| Completed?  Problems Encountered?
+
| Completed. Discussed project design of both hardware and software.
 +
|-
 +
! scope="row"| 2
 +
| 10/31-11/6
 +
| Part testing and project outline
 +
| Completed testing Sharp IR sensor and MIDI controller interface.
 +
|-
 +
! scope="row"| 3
 +
| 11/7-11/13
 +
| Order parts
 +
| Completed order of essential parts including VS1053, short IR sensors,IR jumper wires, and analog to digital converters. Completed software.
 +
|-
 +
! scope="row"| 4
 +
| 11/14-11/20
 +
| Setup all keys with sound.
 +
| Completed playing sounds with key activation.
 +
|-
 +
! scope="row"| 5
 +
| 11/21-11/27
 +
| Volume control and power circuit.
 +
| Completed.
 +
|-
 +
! scope="row"| 6
 +
| 11/28-12/4
 +
| Combine circuits and start mounting system.
 +
| Ordered LEDs and LED driver. Started mounting system onto board.
 +
|-
 +
! scope="row"| 7
 +
| 12/5-12/11
 +
| Build LED circuit and implement software for LEDs. Combine circuits and mount rest of system onto board.
 +
| Completed mounting circuit boards and wiring on 12/15. Final test completed on 12/16.
 +
|-
 +
! scope="row"| 8
 +
| 12/12-12/17
 +
| Final changes and demo
 +
| Completed demo on 12/17.
 
|}
 
|}
  
 
== Parts List & Cost ==
 
== Parts List & Cost ==
Give a simple list of the cost of your project broken down by components. Do not write long stories here.
+
 
 +
{| class="wikitable"
 +
|-
 +
! scope="col"| Quanitity
 +
! scope="col"| Price per part
 +
! scope="col"| Part
 +
! scope="col"| Detail
 +
! scope="col"| Vendor
 +
|-
 +
! scope="row"| 1
 +
| $80
 +
| SJ-One Board
 +
| Microcontroller
 +
| SJSU SCE
 +
|-
 +
! scope="row"| 12
 +
| $13.95
 +
| GP2Y0A41SK0F
 +
| Infrared Proximity Sensor Short Range-Sharp
 +
| SparkFun
 +
|-
 +
! scope="row"| 12
 +
| $1.50
 +
| 3-pin JST
 +
| Infrared Sensor Jumper Wire
 +
| SparkFun
 +
|-
 +
! scope="row"| 6
 +
| $2.30
 +
| MCP3002
 +
| Analog to Digital Converter
 +
| SparkFun
 +
|-
 +
 
 +
! scope="row"| 1
 +
| $19.95
 +
| VS1033D
 +
| MIDI Decoder Breakout Board
 +
| SparkFun
 +
|-
 +
! scope="row"| 1
 +
| $1.95
 +
| LD1117V33
 +
| 3.3 V voltage Regulator
 +
| SparkFun
 +
|-
 +
! scope="row"| 1
 +
| $1.50
 +
| PRT-08032
 +
| 3.5 mm audio jack
 +
| SparkFun
 +
|-
 +
! scope="row"| 2
 +
| $1.25
 +
| PRT-09011
 +
| USB female type A speakers
 +
| SparkFun
 +
|-
 +
! scope="row"| 1
 +
| $13.00
 +
| AL-101
 +
| USB powered speakers
 +
| Amazon
 +
|-
 +
! scope="row"| 24
 +
| $0.04
 +
|
 +
| 150 Ohm resistor
 +
| Fry's
 +
|-
 +
! scope="row"| 12
 +
| $0.202
 +
| WP59EGW/CA
 +
| Red/Green Diffused Common Anode LED
 +
| Mouser
 +
|-
 +
! scope="row"| 1
 +
| $1.48
 +
| TLC6C5912QPWRQ1
 +
| 12 Channel Shift Register LED Driver
 +
| Mouser
 +
|-
 +
! scope="row"| 1
 +
| $5.42
 +
| LCQT-TSSOP20
 +
| TSSOP to 20DIP Socket Adapter
 +
| Mouser
 +
|-
 +
! scope="row"| Total
 +
| $328.38
 +
|
 +
|
 +
|
 +
|-
 +
|}
  
 
== Design & Implementation ==
 
== Design & Implementation ==
The design section can go over your hardware and software design.  Organize this section using sub-sections that go over your design and implementation.
 
  
 
=== Hardware Design ===
 
=== Hardware Design ===
Discuss your hardware design hereShow detailed schematics, and the interface here.
+
    [[Board Schematic]]
 +
[[File:CMPE146 F15 ElectronicPiano TopViewNew.png|framed|boarder|none|text-top|Top view schematic]]
 +
 
 +
The board schematic shown displays the precise measurement of the board as well as where the keys are placed on the board. Measurements were made with hand size taken into account such that the average hand, when centered on one key, will not overlap onto nearby keys and accidentally activate them. Keys are also assigned 0 to 11. This assignment is referenced by the LED numbering and the ADC_Ch signal. LEDs are placed beneath the keys, with red LEDs associated to the top keys and green LEDs associated to the bottom keysThe difference between bottom and top keys is about 5 cm in length, which was created in order to differentiate between choice of key being played between top and bottom keys.
 +
 
 +
 
 +
    [[Power Circuit]]
 +
[[File:CMPE 146 F15 ElectronicPiano PowerCircuit.PNG‎|framed|boarder|none|text-top|Power Circuit schematic]]
 +
 
 +
[[File:CMPE146 F15 ElectronicPiano PowerCircuit.jpg‎|framed|boarder|none|text-top|Power Circuit]]
 +
 
 +
The power circuit can be powered by a USB power supply which distributes the source to the SJ-One Board, to the 3V3 voltage regulator, and to another USB A connector intended to supply power to an external speaker. The 3V3 voltage regulator distributes power to the VS1033D MIDI decoder, to the ADC circuit, and to the LED driver. The power circuit also contains a 3.5mm audio out connector to direct the audio signal from the MIDI decoder to an external speaker; right and left channel are tied together at the connector. Additionally, the board contains an AC coupling circuit to remove the DC offset from the audio signal.
 +
 
 +
 
 +
    [[SJ-One Board]]
 +
[[File:CMPE146 F15 ElectronicPiano NewSJOneBoard.png|framed|boarder|none|text-top|SJ-One Board]]
 +
 
 +
The SJ-One Board is powered by the 5V and ground supplied by the power circuit. The CLK pin is outputted to the VS1033D, analog to digital converters, and LED driver. The MOSI pin is outputted to the VS1033D, analog to digital converters, and LED driver. The MISO pin is only outputted to the analog to digital converters. There are also 8 GPIO output pins used for  chip selecting. P1.30 is used for selecting the VS1033D, while P2.4 is used for selecting the LED driver. Pins P1.29, P1.28, P1.23, P1.22, P1.20, and P1.19 are used for selecting between analog digital converters 0 to 5, respectively. Four GPIO inputs are used for user control. Pins P1.14 and P1.15 are used for lowering and raising the volume, respectively. Pins P1.9 and P1.10 are used for lowering and raising the note selection, respectively. LEDs 0 to 3, which are controlled by GPIO output pins P1.0, P1.1, P1.4, and P1.8, respectively, are used to display current volume, while the 7-segment LEDs represent the note selection.
 +
 +
 
 +
    [[VS1033D Breakout Board]]
 +
[[File:CMPE 146 F15 ElectronicPiano VS1033D Breakout Board.PNG|framed|boarder|left|text-top|VS1033D Breakout Board Schematic]]
 +
 
 +
[[File:CMPE146_F15_ElectronicPiano_VS1033D_BOB.jpg|framed|boarder|center|text-top|VS1033D Breakout Board]]
 +
 
 +
The VS1033D functions as a real-time MIDI decoder; piano notes can be played by sending MIDI standard messages via SPI to the device; audio is obtained from analog output RIGHT. For each MIDI data frame, a byte 0xFF must be sent after. To enable RT MIDI mode, IO1 must be HIGH and IO0 must be LOW on power up. Externally undriven input pins not shown in the schematics, like IO0, have pull-down resistors within the PCB. The chip select CS (active low) is not used in RT MIDI mode; instead, BSYNC functions as chip select. The VS1033D UART controller is not used; Rx must be tied HIGH to remain inactive. Its Serial Out (SO) is left unconnected (labeled as NO_CONNECT), since its returning data is not used.
 +
 
 +
 
 +
    [[Analog to Digital Converter Circuit]]
 +
[[File:CMPE146 F15 ElectronicPiano OneADCNew.png|framed|boarder|left|text-top|Single analog to digital converter]]
 +
 
 +
[[File:CMPE146 F15 ElectronicPiano ADC.png|framed|boarder|center|text-top|ADC Circuit schematic]]
 +
 
 +
[[File:CMPE146_F15_ElectronicPiano_ADC_Circuit.jpg|framed|boarder|none|text-top|ADC Circuit]]
 +
 
 +
As seen, the MOSI, MISO, and CLK connections from the SJ-One Board connect to all of the analog to digital converter's Din, Dout, and CLK pins, respectively. 6 GPIO pins from the SJ-One Board are tied to each of the 6 CS pins of the analog to digital converters. VDD is connected to a 3.3V source. Finally, the channels of the ADC are connected to the Vout of the short IR sensors.
 +
 
 +
 
 +
    [[IR Proximity Sensor]]
 +
[[File:CMPE146_F15_ElectronicPiano_IR_Proximity_Sensor.JPG|framed|boarder|none|text-top|IR Proximity Sensor]]
 +
 
 +
The IR proximity sensor functions as a short-ranged distance sensor. The electronic piano consists of 12 of these sensors; each of the sensor's analog voltage output is connected to a unique ADC channel ADC_CH11 through ADC_CH0 of the Analog to Digital Converter Circuit.
 +
 
 +
 
 +
    [[LED Driver and LEDs]]
 +
[[File:CMPE146 F15 ElectronicPiano LEDdriver.png|framed|boarder|left|text-top|LED driver circuit]]
 +
[[File:CMPE146 F15 ElectronicPiano TopLED.png|framed|boarder|center|text-top|Top key LED]]
 +
[[File:CMPE146 F15 ElectronicPiano BottomLED.png|framed|boarder|center|text-top|Bottom key LED]]
 +
As seen, Drain0 to Drain11 are connected to 150 Ohm resistors, which in turn, are connected to their corresponding numbered key LED cathodes. All LEDs have their anodes connected to a 5 V source. LEDs 0, 1, 2, 3, 4, 5, and 7 have its green cathode (shorter) connect to 150 Ohm resistors and then to ground in order to illuminate green color all the time. Likewise, LEDs 10, 11, 8, 9, and 6 have their red cathodes (longer) connected to 150 Ohm resistors and then to ground in order to illuminate red color at all times. The cathodes are connected to 150 Ohm resistors such that the current going through the LED is about 18 mA for green and 20 mA for red. Drains 0, 1, 2, 3, 4, 5, and 7 are connected to 150 Ohm resistors and then to their corresponding LEDs' red cathode. Drains 10, 11, 8, 9, and 6 are connected to 150 Ohm resistors and then to their corresponding LEDs' green cathode. The GND and G pins are connected to ground, while the VCC and CLR pins are connected to  3.3 V source. The MOSI, CLK, and GPIO output pins from the SJ-One board are connected to the SER IN, SCK, and RCK pins of the LED driver.
  
 
=== Hardware Interface ===
 
=== Hardware Interface ===
In this section, you can describe how your hardware communicates, such as which BUSes used. You can discuss your driver implementation here, such that the '''Software Design''' section is isolated to talk about high level workings rather than inner working of your project.
+
SPI was used to communicate with the 6 ADC devices, the MIDI decoder, and the LED driver. Meanwhile, GPIO was used as control signals, such as chip select and output enable, for the MIDI decoder and the LED driver. In the software, the following SPI and GPIO driver were used to communicate to the IO devices.
 +
 
 +
SPI driver:
 +
    namespace My_spi {
 +
        void init_ssp1_spi( void );                  // Using SCK1, MISO1, MOSI1 (P0.7, P0.8, P0.9)
 +
        void init_cs( uint8_t port, uint8_t pin );
 +
        char spi_exchange_byte( char byte );
 +
        void cs_select( uint8_t port, uint8_t pin );
 +
        void cs_deselect( uint8_t port, uint8_t pin );
 +
    }
 +
 
 +
The init_ssp1_spi() function initializes LPC1758 SSP1 controller to function as an SPI controller with the following setting: MSB first, 8-bit data frame, and 1 MHz clock while other settings remain as default values.
 +
 
 +
The init_cs(port, pin) function initializes a given IO pin to function as a GPIO output used for chip select.
 +
cs_select(port, pin) and cs_deselect(port, pin) functions are used to activate and deactivate a given IO pin respectively.
 +
 
 +
The spi_exchange_byte(byte) function clocks 8-bits to and from a SPI slave device; cs_select(port, pin) and cs_deselect(port, pin) should be called before and after the call of the SPI exchange function(s) to target a specific SPI slave device.
 +
Exception: The LED driver is a shift register and does not use chip select. Instead, a GPIO output is used to signal the shift register to update its output (LEDs). 
 +
 
 +
GPIO driver:
 +
    namespace My_gpio {
 +
        void set_gpio_mode( uint8_t port, uint8_t pin, bool mode );
 +
        void digital_write( uint8_t port, uint8_t pin, bool state );
 +
        bool digital_read( uint8_t port, uint8_t pin );                    // Assuming active high input
 +
    }
 +
 
 +
The set_gpio_mode(port, pin, mode) function initializes a given IO port to function as a specific GPIO mode (mode can be either input or output).
 +
 
 +
The digital_write(port, pin, state) function sets a given GPIO pin to a given voltage level or state (voltage level can either be high or low).
 +
 
 +
The digital_read(port, pin) function reads the voltage level of a given pin and returns a Boolean value (returns true if high and false if low).
  
 
=== Software Design ===
 
=== Software Design ===
Show your software design. For example, if you are designing an MP3 Player, show the tasks that you are using, and what they are doing at a high level. Do not show the details of the code. For example, do not show exact code, but you may show psuedocode and fragments of code. Keep in mind that you are showing DESIGN of your software, not the inner workings of it.
+
[[File:CMPE146 F15 ElectricPaino SoftwareArchitecture.png]]
 +
 
 +
For the software design, there are three tasks: MIDI task (consumer), IR Key task (producer) and Control task (producer). During the idle state, the consumer remains in sleep and waits for a command from the producers. When an action takes place such as an activation of a key or a button, the consumer immediately wakes and executes the received command. Commands and messages are passed using FreeRTOS queues.
 +
 
 +
The MIDI task is responsible for playing notes at a certain pitch and volume. The task also provides feedback of the current pitch (octave) and volume using the SJ-One Board’s 7 segment display and on-board LEDs.
 +
 
 +
The IR Key task is responsible for sampling the IR sensors by reading the values from the ADC. If a key is sampled as active, then the key’s LED activates and the key’s note in addition to the note on command are sent to the MIDI task. The opposite occurs when a key is sampled inactive when previously in the active state such that the key's LED deactivates and the key's note is sent to the MIDI task to be deactivated.
 +
 
 +
The Control task is responsible for reading user input to increment/decrement pitch or volume by sampling the SJ-One Board’s on-board push button. When an input is active, the appropriate command gets sent to the MIDI task.
  
 
=== Implementation ===
 
=== Implementation ===
This section includes implementation, but again, not the details, just the high level. For example, you can list the steps it takes to communicate over a sensor, or the steps needed to write a page of memory onto SPI Flash. You can include sub-sections for each of your component implementation.
+
 
 +
 
 +
MIDI Task Algorithm
 +
 
 +
1. Sleep and wait for a command.
 +
 
 +
2. If the play note command is received, then send the MIDI standard message via SPI to the MIDI decoder to play each note in the Note Queue.
 +
 
 +
3. If the increment/decrement volume command is received, then send the MIDI standard message via SPI to the MIDI decoder to increment/decrement the volume of the MIDI
 +
decoder. 
 +
 
 +
4. If the increment/decrement pitch command is received, then offset base note by one range.
 +
 
 +
5. Return to sleep and wait for another command (repeat from step 1).
 +
 
 +
 
 +
IR Key Sampling and Activation Algorithm
 +
 
 +
1. Using SPI, read 12 ADC channels (6 ADC, 2 channels each) to obtain the values of all 12 IR sensors.
 +
 
 +
2. Compare each value of the 12 IR sensors against valid margin levels.
 +
 
 +
3. If a key meets the valid active margin level, the note of the key is sent to the MIDI task Note’s Queue to be played. (To compensate for noise, a key must be sampled as valid 3 times before being queued to be played.)
 +
 
 +
4. For keys previously in the active state, if its value meets the valid inactive margin level, then the note of the key is sent to the MIDI task Note’s Queue to be turned off.
 +
 
 +
5. After all 12 values are checked, if any note had been sent to the MIDI task’s Note Queue, then update the key’s LED based on its new state.
 +
 
 +
6. Then, after all 12 values are checked AND if any note had been sent to the MIDI task’s Note Queue, send the command Play Note to the MIDI task’s Command Queue.
 +
 
 +
7. Reread ADCs (repeat from step 1).
 +
 
 +
 
 +
Control Task User Input Sampling and Activation Algorithm
 +
 
 +
1. Read the state of the SJ-One Board’s on-board push button.
 +
 
 +
2. If the increment pitch button is sampled active, then send the increment pitch command to the MIDI task; similarly, if the decrement pitch button is sampled active, then
 +
send the decrement pitch command to the MIDI task.
 +
 
 +
3. If the increment volume button is sampled active, then send the increment volume command to the MIDI task; similarly, if the decrement volume button is sampled active, then send the decrement volume command to the MIDI task.
 +
 
 +
4. Reread states of on-board push buttons (repeat from step 1).
  
 
== Testing & Technical Challenges ==
 
== Testing & Technical Challenges ==
Describe the challenges of your project. What advise would you give yourself or someone else if your project can be started from scratch again?
+
Testing was done on individual parts and then on combined parts as part of a circuit. Finally, testing was done on the project as a whole. First, testing was done on the IR sensors in order to read values while moving our hands over it and using the SJ-One Board's analog to digital converter. At the same time, testing was done on the VS1033D by sending MIDI messages to it and playing piano notes through headphones. Next, the IR sensors were attached to the VS1033D in order to activate notes when activated by motion. The 6 external analog to digital converters are then tested for reading input via motion from 12 IR sensors simultaneously. This circuit was then attached to the VS1033D in order to test for activation of different notes with its associated key. Finally, the LED driver was tested separately by sending SPI messages in order to activate specific LEDs. Finally, the LED driver was integrated with the rest of the system; an activation of a key should activate its specific note and LED.
Make a smooth transition to testing section and described what it took to test your project.
 
  
Include sub-sections that list out a problem and solution, such as:
+
=== Issue #1-ADC chip select===
 +
A problem occurred when testing the ADC circuit. During the test, one ADC was tested; the unused ADCs' chip selects remained floating; this resulted in bus contention. The problem was fixed by driving all other unused CS to HIGH (inactive), while the CS of the ADC under test was driven LOW (active).
  
=== My Issue #1 ===
+
=== Issue #2-LED driver soldering===
Discuss the issue and resolution.
+
At first, there was difficulty implementing the LED driver circuit. After soldering the LED driver (shift register) to the TSSOP to DIP20 adapter, a short may had existed and the device was not functioning. After improving the soldering, the device functioned properly; the expected LEDs were turned on based on the data sent to it.
 +
 
 +
=== Issue #3-Noisy power supply===
 +
Powering 12 IR proximity sensors introduced noise in the 5V supply line; for this reason, there are three major problems.
 +
 
 +
For the first problem, the voltage of the 5V supply drops to approximately 4.5V-4.2V; supplying inadequate voltage to the system can lead to unpredictable results. To mitigate the problem, a higher-quality power supply was used to reduce the noise and voltage drop. Additionally, a 470uF bypass capacitor was added to the power circuit to improve the noise filtration.
 +
 
 +
For the second problem, false key activation can occur; infrequently, random keys would unintentionally activate without any action from the user. To mitigate the problem, IR debouncing was implemented in the software. Similar to push button debouncing, an IR key must be sampled valid active multiple times in order to play its note.
 +
 
 +
For the third problem, an external speaker that uses the 5V USB connector of the power circuit as a source would amplify the noise and idly emit high pitch static. To mitigate the problem, an external speaker with its own power supply was used to partially isolate the noise in the supply line from the audio. Using this approach (although not an ideal solution), the high pitch static was significantly reduced. Since the source of the noise is from the array of IR sensors, a cost effective solution is to use a passive filter to capture and isolate the noise between the power circuit and the ADC circuit.
 +
 
 +
[[File:CMPE146_F15_ElectronicPiano_Noise_5V_Supply.jpg|framed|boarder|none|text-top|Noise in the 5V supply line]]
  
 
== Conclusion ==
 
== Conclusion ==
Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?
+
This project was completed successfully. The project began with a simple idea of creating a motion controlled electronic piano. Next, we thought of how this would be built and what parts were needed in order to complete this task. We were able to theoretically build the entire system and determine how it functioned through FreeRTOS queues. Once parts were determined, testing was done for each individual part in order to verify its function. Once parts were successfully tested, they would be combine in order to create a bigger system, and eventually, the entire piano system. From the project, we were able to learn how to implement the LPC1758's serial communications in order to build an embedded system. The implementation involved both hardware and software capabilities, which ultimately led to the creation of the electronic piano.
 
 
 
=== Project Video ===
 
=== Project Video ===
Upload a video of your project and post the link here.
+
https://youtu.be/DqBue8DKsk0
  
 
=== Project Source Code ===
 
=== Project Source Code ===
 
*  [https://sourceforge.net/projects/sjsu/files/CmpE_F2015/ Sourceforge Source Code Link]
 
*  [https://sourceforge.net/projects/sjsu/files/CmpE_F2015/ Sourceforge Source Code Link]
 +
 +
[[Media:CMPE146_F15_ElectronicPiano_Electronic_piano_tasks.hpp]]
 +
 +
[[Media:CMPE146_F15_ElectronicPiano_Electronic_piano_tasks.cpp]]
 +
 +
[[Media:CMPE146_F15_ElectronicPiano_My_spi.hpp]]
 +
 +
[[Media:CMPE146_F15_ElectronicPiano_My_spi.cpp]]
 +
 +
[[Media:CMPE146_F15_ElectronicPiano_My_gpio.hpp]]
 +
 +
[[Media:CMPE146_F15_ElectronicPiano_My_gpio.cpp]]
 +
 +
[[Media:CMPE146_F15_ElectronicPiano_main.cpp]]
  
 
== References ==
 
== References ==
 
=== Acknowledgement ===
 
=== Acknowledgement ===
Any acknowledgement that you may wish to provide can be included here.
+
Special acknowledgement to Preetpal Kang for suggesting IR proximity sensors in place of basic image capturing motion sensors.
  
 
=== References Used ===
 
=== References Used ===
List any references used in project.
+
Analog to digital converter datasheet
 +
[[Media: CMPE146_F15_ElectricPiano_ADC.pdf]]
 +
 
 +
Audio jack data sheet
 +
[[Media:CMPE146_F15_ElectricPiano_AudioJack.pdf]]
 +
 
 +
LED data sheet
 +
[[Media:CMPE146_F15_ElectricPiano_LED.pdf]]
 +
 
 +
LED driver data sheet
 +
[[Media:CMPE146_F15_ElectricPiano_LEDdriver.pdf]]
 +
 
 +
Short IR data sheet
 +
[[Media: CMPE146_F15_ElectricPiano_shortIR.pdf]]
 +
 
 +
USB connector data sheet
 +
[[Media:CMPE146_F15_ElectricPiano_USB.pdf]]
 +
 
 +
VS1033D data sheet
 +
[[Media:CMPE_146_F15_ElectronicPiano_VS1033D.pdf]]
 +
 
 +
MIDI Standard Messages
 +
http://www.midi.org/techspecs/midimessages.php
 +
 
 +
MIDI note to Hexadecimal
 +
http://www.wavosaur.com/download/midi-note-hex.php
  
 
=== Appendix ===
 
=== Appendix ===
You can list the references you used.
 

Latest revision as of 17:56, 19 December 2015

Motion Controlled Electronic Piano

Top view of implemented board
Piano keys representation of 1 key short of an octave
Side view of implemented board
Top keys being played
Bottom keys being played



Abstract

The purpose of this project is to design and build a motion controlled electronic piano powered by an SJ-One Board. Users will be able to play the notes of a piano using IR proximity sensors as keys. At activation, the key's LED will light up. The number of keys of the piano will cover one key short of an octave; however, the piano provides users the capability to change (offset) the notes across other ranges.

Objectives & Introduction

The objective of this project is to design and build a multi-keyed motion controlled electronic piano. The piano will feature 12 IR proximity sensors to act as motion activated keys, a real-time MIDI decoder to play piano notes, and LEDs that activate along with the activation of its associated key. The electronic piano will utilize FreeRTOS multitasking environment to simultaneously sample values from multiple IR proximity sensors, analyze the obtained values, then play piano notes according to the active state of the IR keys.

1. Read an IR sensor's value using an external ADC

2. Write a software to read multiple IR sensors and play different piano notes using a MIDI decoder

3. Assemble a range of IR sensors to function as piano keys and an array of ADCs to read all IR sensors

4. Build a wooden frame and mount the control circuits and IR sensors

5. Integrate LEDs to each key and control the LEDs using an LED driver

Team Members & Responsibilities

  • Jason Tran
    • Design and write the software
    • Design and build the power circuit and ADC circuit
  • Arthur Nguyen
    • Design and build the wooden frame
    • Mount boards and IR keys onto the wooden frame
    • Design and build the LED circuit

Schedule

Week# Date Task Actual
1 10/24-10/30 Project proposal Completed. Discussed project design of both hardware and software.
2 10/31-11/6 Part testing and project outline Completed testing Sharp IR sensor and MIDI controller interface.
3 11/7-11/13 Order parts Completed order of essential parts including VS1053, short IR sensors,IR jumper wires, and analog to digital converters. Completed software.
4 11/14-11/20 Setup all keys with sound. Completed playing sounds with key activation.
5 11/21-11/27 Volume control and power circuit. Completed.
6 11/28-12/4 Combine circuits and start mounting system. Ordered LEDs and LED driver. Started mounting system onto board.
7 12/5-12/11 Build LED circuit and implement software for LEDs. Combine circuits and mount rest of system onto board. Completed mounting circuit boards and wiring on 12/15. Final test completed on 12/16.
8 12/12-12/17 Final changes and demo Completed demo on 12/17.

Parts List & Cost

Quanitity Price per part Part Detail Vendor
1 $80 SJ-One Board Microcontroller SJSU SCE
12 $13.95 GP2Y0A41SK0F Infrared Proximity Sensor Short Range-Sharp SparkFun
12 $1.50 3-pin JST Infrared Sensor Jumper Wire SparkFun
6 $2.30 MCP3002 Analog to Digital Converter SparkFun
1 $19.95 VS1033D MIDI Decoder Breakout Board SparkFun
1 $1.95 LD1117V33 3.3 V voltage Regulator SparkFun
1 $1.50 PRT-08032 3.5 mm audio jack SparkFun
2 $1.25 PRT-09011 USB female type A speakers SparkFun
1 $13.00 AL-101 USB powered speakers Amazon
24 $0.04 150 Ohm resistor Fry's
12 $0.202 WP59EGW/CA Red/Green Diffused Common Anode LED Mouser
1 $1.48 TLC6C5912QPWRQ1 12 Channel Shift Register LED Driver Mouser
1 $5.42 LCQT-TSSOP20 TSSOP to 20DIP Socket Adapter Mouser
Total $328.38

Design & Implementation

Hardware Design

   Board Schematic
Top view schematic

The board schematic shown displays the precise measurement of the board as well as where the keys are placed on the board. Measurements were made with hand size taken into account such that the average hand, when centered on one key, will not overlap onto nearby keys and accidentally activate them. Keys are also assigned 0 to 11. This assignment is referenced by the LED numbering and the ADC_Ch signal. LEDs are placed beneath the keys, with red LEDs associated to the top keys and green LEDs associated to the bottom keys. The difference between bottom and top keys is about 5 cm in length, which was created in order to differentiate between choice of key being played between top and bottom keys.


    Power Circuit
Power Circuit schematic
Power Circuit

The power circuit can be powered by a USB power supply which distributes the source to the SJ-One Board, to the 3V3 voltage regulator, and to another USB A connector intended to supply power to an external speaker. The 3V3 voltage regulator distributes power to the VS1033D MIDI decoder, to the ADC circuit, and to the LED driver. The power circuit also contains a 3.5mm audio out connector to direct the audio signal from the MIDI decoder to an external speaker; right and left channel are tied together at the connector. Additionally, the board contains an AC coupling circuit to remove the DC offset from the audio signal.


    SJ-One Board
SJ-One Board

The SJ-One Board is powered by the 5V and ground supplied by the power circuit. The CLK pin is outputted to the VS1033D, analog to digital converters, and LED driver. The MOSI pin is outputted to the VS1033D, analog to digital converters, and LED driver. The MISO pin is only outputted to the analog to digital converters. There are also 8 GPIO output pins used for chip selecting. P1.30 is used for selecting the VS1033D, while P2.4 is used for selecting the LED driver. Pins P1.29, P1.28, P1.23, P1.22, P1.20, and P1.19 are used for selecting between analog digital converters 0 to 5, respectively. Four GPIO inputs are used for user control. Pins P1.14 and P1.15 are used for lowering and raising the volume, respectively. Pins P1.9 and P1.10 are used for lowering and raising the note selection, respectively. LEDs 0 to 3, which are controlled by GPIO output pins P1.0, P1.1, P1.4, and P1.8, respectively, are used to display current volume, while the 7-segment LEDs represent the note selection.


    VS1033D Breakout Board
VS1033D Breakout Board Schematic
VS1033D Breakout Board

The VS1033D functions as a real-time MIDI decoder; piano notes can be played by sending MIDI standard messages via SPI to the device; audio is obtained from analog output RIGHT. For each MIDI data frame, a byte 0xFF must be sent after. To enable RT MIDI mode, IO1 must be HIGH and IO0 must be LOW on power up. Externally undriven input pins not shown in the schematics, like IO0, have pull-down resistors within the PCB. The chip select CS (active low) is not used in RT MIDI mode; instead, BSYNC functions as chip select. The VS1033D UART controller is not used; Rx must be tied HIGH to remain inactive. Its Serial Out (SO) is left unconnected (labeled as NO_CONNECT), since its returning data is not used.


    Analog to Digital Converter Circuit
Single analog to digital converter
ADC Circuit schematic
ADC Circuit

As seen, the MOSI, MISO, and CLK connections from the SJ-One Board connect to all of the analog to digital converter's Din, Dout, and CLK pins, respectively. 6 GPIO pins from the SJ-One Board are tied to each of the 6 CS pins of the analog to digital converters. VDD is connected to a 3.3V source. Finally, the channels of the ADC are connected to the Vout of the short IR sensors.


    IR Proximity Sensor
IR Proximity Sensor

The IR proximity sensor functions as a short-ranged distance sensor. The electronic piano consists of 12 of these sensors; each of the sensor's analog voltage output is connected to a unique ADC channel ADC_CH11 through ADC_CH0 of the Analog to Digital Converter Circuit.


    LED Driver and LEDs
LED driver circuit
Top key LED
Bottom key LED

As seen, Drain0 to Drain11 are connected to 150 Ohm resistors, which in turn, are connected to their corresponding numbered key LED cathodes. All LEDs have their anodes connected to a 5 V source. LEDs 0, 1, 2, 3, 4, 5, and 7 have its green cathode (shorter) connect to 150 Ohm resistors and then to ground in order to illuminate green color all the time. Likewise, LEDs 10, 11, 8, 9, and 6 have their red cathodes (longer) connected to 150 Ohm resistors and then to ground in order to illuminate red color at all times. The cathodes are connected to 150 Ohm resistors such that the current going through the LED is about 18 mA for green and 20 mA for red. Drains 0, 1, 2, 3, 4, 5, and 7 are connected to 150 Ohm resistors and then to their corresponding LEDs' red cathode. Drains 10, 11, 8, 9, and 6 are connected to 150 Ohm resistors and then to their corresponding LEDs' green cathode. The GND and G pins are connected to ground, while the VCC and CLR pins are connected to 3.3 V source. The MOSI, CLK, and GPIO output pins from the SJ-One board are connected to the SER IN, SCK, and RCK pins of the LED driver.

Hardware Interface

SPI was used to communicate with the 6 ADC devices, the MIDI decoder, and the LED driver. Meanwhile, GPIO was used as control signals, such as chip select and output enable, for the MIDI decoder and the LED driver. In the software, the following SPI and GPIO driver were used to communicate to the IO devices.

SPI driver:

   namespace My_spi {
       void init_ssp1_spi( void );                   // Using SCK1, MISO1, MOSI1 (P0.7, P0.8, P0.9)
       void init_cs( uint8_t port, uint8_t pin );
       char spi_exchange_byte( char byte );
       void cs_select( uint8_t port, uint8_t pin );
       void cs_deselect( uint8_t port, uint8_t pin );
   }

The init_ssp1_spi() function initializes LPC1758 SSP1 controller to function as an SPI controller with the following setting: MSB first, 8-bit data frame, and 1 MHz clock while other settings remain as default values.

The init_cs(port, pin) function initializes a given IO pin to function as a GPIO output used for chip select. cs_select(port, pin) and cs_deselect(port, pin) functions are used to activate and deactivate a given IO pin respectively.

The spi_exchange_byte(byte) function clocks 8-bits to and from a SPI slave device; cs_select(port, pin) and cs_deselect(port, pin) should be called before and after the call of the SPI exchange function(s) to target a specific SPI slave device. Exception: The LED driver is a shift register and does not use chip select. Instead, a GPIO output is used to signal the shift register to update its output (LEDs).

GPIO driver:

   namespace My_gpio {
       void set_gpio_mode( uint8_t port, uint8_t pin, bool mode );
       void digital_write( uint8_t port, uint8_t pin, bool state );
       bool digital_read( uint8_t port, uint8_t pin );                     // Assuming active high input
   }

The set_gpio_mode(port, pin, mode) function initializes a given IO port to function as a specific GPIO mode (mode can be either input or output).

The digital_write(port, pin, state) function sets a given GPIO pin to a given voltage level or state (voltage level can either be high or low).

The digital_read(port, pin) function reads the voltage level of a given pin and returns a Boolean value (returns true if high and false if low).

Software Design

CMPE146 F15 ElectricPaino SoftwareArchitecture.png

For the software design, there are three tasks: MIDI task (consumer), IR Key task (producer) and Control task (producer). During the idle state, the consumer remains in sleep and waits for a command from the producers. When an action takes place such as an activation of a key or a button, the consumer immediately wakes and executes the received command. Commands and messages are passed using FreeRTOS queues.

The MIDI task is responsible for playing notes at a certain pitch and volume. The task also provides feedback of the current pitch (octave) and volume using the SJ-One Board’s 7 segment display and on-board LEDs.

The IR Key task is responsible for sampling the IR sensors by reading the values from the ADC. If a key is sampled as active, then the key’s LED activates and the key’s note in addition to the note on command are sent to the MIDI task. The opposite occurs when a key is sampled inactive when previously in the active state such that the key's LED deactivates and the key's note is sent to the MIDI task to be deactivated.

The Control task is responsible for reading user input to increment/decrement pitch or volume by sampling the SJ-One Board’s on-board push button. When an input is active, the appropriate command gets sent to the MIDI task.

Implementation

MIDI Task Algorithm

1. Sleep and wait for a command.

2. If the play note command is received, then send the MIDI standard message via SPI to the MIDI decoder to play each note in the Note Queue.

3. If the increment/decrement volume command is received, then send the MIDI standard message via SPI to the MIDI decoder to increment/decrement the volume of the MIDI decoder.

4. If the increment/decrement pitch command is received, then offset base note by one range.

5. Return to sleep and wait for another command (repeat from step 1).


IR Key Sampling and Activation Algorithm

1. Using SPI, read 12 ADC channels (6 ADC, 2 channels each) to obtain the values of all 12 IR sensors.

2. Compare each value of the 12 IR sensors against valid margin levels.

3. If a key meets the valid active margin level, the note of the key is sent to the MIDI task Note’s Queue to be played. (To compensate for noise, a key must be sampled as valid 3 times before being queued to be played.)

4. For keys previously in the active state, if its value meets the valid inactive margin level, then the note of the key is sent to the MIDI task Note’s Queue to be turned off.

5. After all 12 values are checked, if any note had been sent to the MIDI task’s Note Queue, then update the key’s LED based on its new state.

6. Then, after all 12 values are checked AND if any note had been sent to the MIDI task’s Note Queue, send the command Play Note to the MIDI task’s Command Queue.

7. Reread ADCs (repeat from step 1).


Control Task User Input Sampling and Activation Algorithm

1. Read the state of the SJ-One Board’s on-board push button.

2. If the increment pitch button is sampled active, then send the increment pitch command to the MIDI task; similarly, if the decrement pitch button is sampled active, then send the decrement pitch command to the MIDI task.

3. If the increment volume button is sampled active, then send the increment volume command to the MIDI task; similarly, if the decrement volume button is sampled active, then send the decrement volume command to the MIDI task.

4. Reread states of on-board push buttons (repeat from step 1).

Testing & Technical Challenges

Testing was done on individual parts and then on combined parts as part of a circuit. Finally, testing was done on the project as a whole. First, testing was done on the IR sensors in order to read values while moving our hands over it and using the SJ-One Board's analog to digital converter. At the same time, testing was done on the VS1033D by sending MIDI messages to it and playing piano notes through headphones. Next, the IR sensors were attached to the VS1033D in order to activate notes when activated by motion. The 6 external analog to digital converters are then tested for reading input via motion from 12 IR sensors simultaneously. This circuit was then attached to the VS1033D in order to test for activation of different notes with its associated key. Finally, the LED driver was tested separately by sending SPI messages in order to activate specific LEDs. Finally, the LED driver was integrated with the rest of the system; an activation of a key should activate its specific note and LED.

Issue #1-ADC chip select

A problem occurred when testing the ADC circuit. During the test, one ADC was tested; the unused ADCs' chip selects remained floating; this resulted in bus contention. The problem was fixed by driving all other unused CS to HIGH (inactive), while the CS of the ADC under test was driven LOW (active).

Issue #2-LED driver soldering

At first, there was difficulty implementing the LED driver circuit. After soldering the LED driver (shift register) to the TSSOP to DIP20 adapter, a short may had existed and the device was not functioning. After improving the soldering, the device functioned properly; the expected LEDs were turned on based on the data sent to it.

Issue #3-Noisy power supply

Powering 12 IR proximity sensors introduced noise in the 5V supply line; for this reason, there are three major problems.

For the first problem, the voltage of the 5V supply drops to approximately 4.5V-4.2V; supplying inadequate voltage to the system can lead to unpredictable results. To mitigate the problem, a higher-quality power supply was used to reduce the noise and voltage drop. Additionally, a 470uF bypass capacitor was added to the power circuit to improve the noise filtration.

For the second problem, false key activation can occur; infrequently, random keys would unintentionally activate without any action from the user. To mitigate the problem, IR debouncing was implemented in the software. Similar to push button debouncing, an IR key must be sampled valid active multiple times in order to play its note.

For the third problem, an external speaker that uses the 5V USB connector of the power circuit as a source would amplify the noise and idly emit high pitch static. To mitigate the problem, an external speaker with its own power supply was used to partially isolate the noise in the supply line from the audio. Using this approach (although not an ideal solution), the high pitch static was significantly reduced. Since the source of the noise is from the array of IR sensors, a cost effective solution is to use a passive filter to capture and isolate the noise between the power circuit and the ADC circuit.

Noise in the 5V supply line

Conclusion

This project was completed successfully. The project began with a simple idea of creating a motion controlled electronic piano. Next, we thought of how this would be built and what parts were needed in order to complete this task. We were able to theoretically build the entire system and determine how it functioned through FreeRTOS queues. Once parts were determined, testing was done for each individual part in order to verify its function. Once parts were successfully tested, they would be combine in order to create a bigger system, and eventually, the entire piano system. From the project, we were able to learn how to implement the LPC1758's serial communications in order to build an embedded system. The implementation involved both hardware and software capabilities, which ultimately led to the creation of the electronic piano.

Project Video

https://youtu.be/DqBue8DKsk0

Project Source Code

Media:CMPE146_F15_ElectronicPiano_Electronic_piano_tasks.hpp

Media:CMPE146_F15_ElectronicPiano_Electronic_piano_tasks.cpp

Media:CMPE146_F15_ElectronicPiano_My_spi.hpp

Media:CMPE146_F15_ElectronicPiano_My_spi.cpp

Media:CMPE146_F15_ElectronicPiano_My_gpio.hpp

Media:CMPE146_F15_ElectronicPiano_My_gpio.cpp

Media:CMPE146_F15_ElectronicPiano_main.cpp

References

Acknowledgement

Special acknowledgement to Preetpal Kang for suggesting IR proximity sensors in place of basic image capturing motion sensors.

References Used

Analog to digital converter datasheet Media: CMPE146_F15_ElectricPiano_ADC.pdf

Audio jack data sheet Media:CMPE146_F15_ElectricPiano_AudioJack.pdf

LED data sheet Media:CMPE146_F15_ElectricPiano_LED.pdf

LED driver data sheet Media:CMPE146_F15_ElectricPiano_LEDdriver.pdf

Short IR data sheet Media: CMPE146_F15_ElectricPiano_shortIR.pdf

USB connector data sheet Media:CMPE146_F15_ElectricPiano_USB.pdf

VS1033D data sheet Media:CMPE_146_F15_ElectronicPiano_VS1033D.pdf

MIDI Standard Messages http://www.midi.org/techspecs/midimessages.php

MIDI note to Hexadecimal http://www.wavosaur.com/download/midi-note-hex.php

Appendix