Difference between revisions of "F19: M&B (Morph & Blend)"

From Embedded Systems Learning Academy
Jump to: navigation, search
(Testing & Technical Challenges)
(Testing & Technical Challenges)
Line 565: Line 565:
  
 
== Testing & Technical Challenges ==
 
== Testing & Technical Challenges ==
IMPLEMENTING CONTROL FOR 32 x 64 LED MATRIX DISPLAY
+
'''''Developing a driver for the LED matrix...'''''<br>
  
After an initial understanding of how to control the LED matrix, we first developed a code to light up a pixel and then display a character. We first implemented a function to first select the row according to A, B,C and D signals and write the corresponding pixel values of all LEDs in those rows with the updated values from the frame buffer. The values of the pixels were set only within the clock trigger. In the end, the pixel values were latched before the output enable signal was asserted again. This function was called from a periodic task with the 1 millisecond delay in order to get the refresh rate of the display as 1 millisecond. This helped us to get control over individual pixels of the matrix.
+
The biggest technical challenge that we faced was interfacing the LPC 1758 micro-controller, with the Adafruit 32 x 64 LED matrix.  
  
PREVENTING LEFT SHIFT OF THE PLAYER
+
Adafruit (the 32 x 64 LED matrix's manufacturer) provides a basic tutorial on their website, in order to help with interfacing the LED matrix to a micro-controller. The tutorial briefly describes the functions of the pins on the board's input and output connector ports. It also describes how to connect jumper wires to the ribbon cable (which comes packaged with the board), which connects to the matrix's input and output connector ports. It assumes that the user will utilize an Arduino-type micro-controller to drive the matrix's RGB LED's.
  
The shift of obstacles led to the shifting of the player pixels too. This was avoided by drawing and clearing the player in the update display task which is the highest priority task happening once every 2 ms.
+
Because we used the LPC 1758 to drive the LED matrix, our choices were to either adapt a third party driver library designed for an Arduino, or develop a new driver. We chose the latter option, as the third party Arduino-focused API's were difficult to fully understand. We also felt that developing our own driver would allow us to gain a strong understanding of all aspects of micro-controller to LED matrix integration. This proved useful both for firmware development and debugging.
  
 +
The first part of developing the LED matrix driver, was learning how each of the 16 input pins affected the matrix display.
 +
 +
 +
The LED matrix's input socket looked as follows:
 +
 +
[[File:Adafruit_32x32_LED_Matrix_socket.png | 300x350px]]
 +
 +
 +
 +
'''''GND...'''''<br>
 +
The 3 '''GND''' pins were the easiest to understand. They provided a ground signal reference for the matrix's RGB LED's. The '''GND''' pins on the LED matrix were connected to '''GND''' on the LPC 1758, through a ground plane on the PCB, allowing the micro-controller and LED matrix to share the same signal reference.
 +
 +
 +
 +
'''''A B C D...'''''<br>
 +
Pins '''A''', '''B''', '''C''' and '''D''', were used as control bits for the LED matrix's row multiplexer. The 32 x 64 matrix is made up of two individual 16 x 64 LED panels. Each panel contains 16 rows and 32 columns. The binary values of the multiplexer signals, '''A''', '''B''', '''C''' and '''D''',  are used to determine which row of LED's is being driven. Each panel drives the same row of LED's at a time, based on these values. For example, if  '''A''', '''B''', '''C''' and '''D''' are set to 0000, then the LED's in the first row of each panel can be driven. If  '''A''', '''B''', '''C''' and '''D''' are set to 0001, then the LED's in the second row of each panel can be driven. This pattern continues until  '''A''', '''B''', '''C''' and '''D''' are set to 1111, which corresponds to sixteenth (bottom) row of each panel. We set our row iterating value, as '''i = (i + 1) % 16''', so that  '''A''', '''B''', '''C''' and '''D''', would be set to 0000, after finishing latching data into the sixteenth row. This allowed the multiplexer to continuously loop through each row in each panel, from top to bottom. We applied a delay of 1ms after each row iteration, thereby setting the overall scan rate to 62Hz. This scan rate was too fast for the human eye to detect, which allowed the LED matrix to display objects steadily, without annoying flickering compromising the image quality.
 +
 +
 +
 +
'''''R1 B1 G1'''''<br>
 +
'''''R2 G2 B2...'''''<br>
 +
Pins '''R1''', '''B1''', '''G1''' and '''R2''', '''B2''', '''G2''', were the 6 control signals used to drive the matrix's RGB LEDs, different combinations of red, blue and green (depending on their binary values). '''R1''', '''B1''' and '''G1''', controlled indiidual LED colors on the top panel (panel 1), while '''R2''', '''B2''' and '''G2''', controlled individual LED colors on the bottom panel (panel 2). The way these color combinations manifested on the board, were as follows:
 +
 +
{| class="wikitable" style="text-align: center; width: 100px; height: 100px;"
 +
|-
 +
|  '''R'''  || '''B'''  || '''G''' || '''COLOR'''
 +
|-
 +
| 0 || 0 || 0 ||  '''OFF'''
 +
|-
 +
| 0 || 0 || 1 ||  '''GREEN'''
 +
|-
 +
| 0 || 1 || 0 ||  '''BLUE'''
 +
|-
 +
| 0 || 1 || 1 ||  '''CYAN'''
 +
|-
 +
| 1 || 0 || 0 ||  '''RED'''
 +
|-
 +
| 1 || 0 || 1 ||  '''YELLOW'''
 +
|-
 +
| 1 || 1 || 0 ||  '''PURPLE'''
 +
|-
 +
| 1 || 1 || 1 ||  '''WHITE'''
 +
|}
 +
   
 +
In order to understand how these signals drive individual LEDs on the matrix, one must understand the hardware that they control. The matrix's RGB LEDs are driven by 6 column drivers, each having 32 outputs. Each panel has 3 column drivers. Each column driver can drive 32 LED's in each panel, either red, green or blue (because each of the 3 column drivers controls one of these 3 colors). As a result, the 6 column drivers can drive up to 192 individual red, green and blue LED's (each RGB LED consists of 3 individual red, green and blue LED's) at a time.
 +
 +
 +
'''''CLK LAT OE...'''''<br>
 +
Each column driver is composed of a serial data input, a blanking input, a shift register, and a parallel output register. Data bits are passed into the shift register on the rising edge of each clock cycle. The clock is controlled using the signal '''CLK'''. Because there are 32 RGB LED's per row, 32 clock cycles must occur, in order to drive all LED's in a row. After 32 bits of data are passed into the shift register, the '''LATCH''' signal (which is set low before shifting in data bits to the shift register) must be asserted, in order to transfer the data bits from the shift register to the parallel output register. Output enable ('''OE''') is pulled low at this point, in order to enable the column driver. It is pulled high again, when transitioning to a new row.
 +
 +
[[File:CMPE244 F18 T4 Rgb-led-panel-display-organization.png | 500x500px]] [[File:CMPE244 F18 T4 Rgb-led-panel-shift-register.png | 500x600px]]
 +
 +
We designed our driver to operate on 32 bit, bit strings (of type uint32_t), in order to utilize all 32 LEDs in each row of the matrix. Each 32 bit string, was stored in a 32 element array of type, uint32_t. We assigned the base memory address of the array to a pointer, and used integer offset values to pass the different bit strings, stored in the array, into the matrix driver function. This method allowed us to pass 32 unique bit strings into all 32 rows of the LED matrix. The integer offsets corresponded to row numbers. For example an offset of 4, would increment the pointer, pointing at the memory address of row 0, by 4 memory addresses. Dereferencing the pointer with this offset (4), would give access to data bits governing the output of row 4 of the LED matrix (the fifth row).
 +
 +
For each row, we used a for loop (which iterated from 0 to 31) in order to shift all 192 data bits into the shift registers. We latched the data bits into the parallel output register, after shifting was completed, which allowed us to illuminate different individual LED's in each row. This allowed us to use 32 bit, bit strings, to create data structures and graphics.
 +
 +
The column driver also allowed us to give the illusion of motion, by turning different LED's on and off in adjacent rows.
 +
 +
 +
'''''Integrating an acceleration sensor to move an avatar...'''''<br>
 +
 +
Another significant technical challenge that we faced, was integrating the LPC 1758's acceleration sensor with a virtual avatar (the player sprite). We used feedback from the acceleration sensor's Y-axis tilt output, to govern the player sprite's lateral movement. In this day, the play had one dimension of control over the sprite, by adjusting the Y-axis tilt of the LPC 1758.
 +
 +
While the task using acceleration sensor feedback to control movement was fairly trivial, it induced a bug in the game. The player sprite flickered one column to the left or one column to the right intermittently, when moving. When the LPC 1758 was completely still, flickering did not occur.
 +
 +
We tried sampling the acceleration sensor 10 times and averaging the integer result, before scaling it by our chosen step value (83), in order to apply the appropriate bit shift to the player sprite. This did not resolve the flickering.
 +
 +
We also tried processing the acceleration Y-axis feedback in a separate task from where we drew the sprite. We included a 1ms delay for each task, as we were concerned that there may have been overlap of calculating acceleration sensor feedback and applying the bit shift to the player sprite.
 +
 +
A few modifications to our move_player_sprite() function led to a significant improvement in performance. We increased the acceleration sensor step size (which is used to scale the Y-axis feedback integer value into a bit shift offset) and implemented our software interpretation of bandpass filter. We also stored the difference between the previous acceleration sensor Y-axis feedback value and the current value in a variable. We used the absolute value of this difference to determine if the acceleration sensor had registered significant enough movement to justify changing the current sprite position offset value. We also increased the move_player_sprite() function's block time to 100ms, which allowed it to more accurately track real-time movement (as human hands don't move fast enough to justify a 1ms delay). These modifications significantly reduced player sprite flickering.
 +
 +
 +
'''''Tracking a moving sprite and integrating shooting...'''''<br>
 +
Tracking the player sprite was a fairly trivial task, due to our player sprite only moving in one dimension (left and right). Because the player sprite was holding a 1 pixel wide gun, we continuously scanned the row where the gun was located, in order to determine which column of the LED matrix would be set as the projectile path. We constantly scanned for a data bit set to 1, which when found, would let us know exactly where in the row that the gun was located.
 +
 +
A "shoot" signal was triggered by a push button sensor, which unblocked the "fire_a_shot()" function. This function passed the gun's column location to the "projectile_path()" function, allowing the projectile sprite to travel vertically, towards a moving enemy sprite (or nothing at all).
 +
 +
 +
'''''Enemy sprites and re-spawn time...'''''<br>
 +
We created 5 unique enemy sprites, using combinations of adjacent bit strings. Each enemy sprite traversed the LED matrix laterally and was managed by its own task. We used a random number generator in addition with a fixed-time modulus, in order to implement random re-spawn times of sprites. The enemy sprite's position in the virtual LED matrix array was incremented by one column, after a set time delay. Each enemy sprite had a unique time delay, in order to allow some sprites to move faster than others and thus be harder to shoot. Faster sprites were worth more points (when shot), while slower sprites, while easier to hit, resulted in less points added to the player's total game score.
 +
 +
 +
'''''Hit detection and scorekeeping...'''''<br>
 +
Hit detection proved less challenging that we initially thought that it would be. When a projectile was fired at an enemy sprite, the projectile's position was tracked as it traversed the LED matrix, vertically. When the projectile's position was within one pixel of the enemy sprite (and directly beneath it), a hit was registered. Upon a hit, the all data bits in the rows containing the vanquished enemy sprite were cleared to "0", This effectively removed the enemy sprite from the game, until a new identical enemy sprite re-spawned. We incorporated a random number driven delay, to control the re-spawn time of enemy sprites. This random re-spawn time was applied upon hits, as well as if an enemy sprite successfully traversed all 32 columns of the matrix (without being hit by a projectile).
 +
 +
 +
'''''Start Procedure...'''''<br>
 +
In order to create a "Start" message for the game (essentially an introduction to the game), we created a task function called "Start_Game()". This task loaded data bits comprising a starting message, into the  virtual LED matrix array. It also used the unique task handles of the in-game task functions, in order to suspend them, while it ran. After a fixed delay of 10 seconds, All data bits in the virtual LED matrix were set to 0; creating a blank screen. Start_Game() then used the task handles of each in-game task, to resume them. This commenced the actual Spartan Warrior game. We also sent a high (1) signal to the other LPC 1758 micro-controller in the system, which interfaced with the mp3 decoder. Upon receiving this signal, the other micro-controller worked with the mp3 decoder to play the soundtrack.
 +
 +
 +
'''''Game Over Procedure...'''''<br>
 +
In order to implement the "Game Over" message, we began by setting a delay of 60 seconds. This allowed the task to block for one minute (or the length of the actual in-game time). We adjusted the input integer parameter of the delay_ms() function, in order to ensure that 60 seconds of actual game time was allowed, before this task was called. We chose this value for the game time, for demo purposes only. We could have adjusted it to any value we wished. Once the delay time expired, all in-game tasks were suspended and all data bits in the virtual LED matrix array, were set to 0. The message "Game Over", was loaded into the array and displayed on the screen, as well as the player's score total (on the lower panel).
 +
 +
 +
'''''MP3 Decoder...'''''
 +
The system uses two LPC1758 boards; one driving the LED matrix display and one, which reads mp3 data from an SD card using SPI. The two boards communicate with each other using a GPIO pin. A binary signal (high) indicates when to start the soundtrack. It is sent during the start_screen() task. The game_over() task pulls the signal low when it unblocks, signalling the LPC 1758, playing the soundtrack, to stop outputting the audio signal.
 +
 +
An audio decoder was used to collect the data from the queue, translate it and play it on the speaker, connected at the output. When the game reaches the end, the GPIO pin is reset which signals the decoder to stop retrieving data from the queue. The code uses the following to get the data from the SD Card and insert it in the queue:
 +
 +
[[Storage::read("1:track1.mp3", &dataChunk[0], 4096, songoffset);
 +
''songoffset is a data member that stores the starting address of 4k bytes to send to the queue.'']]
 +
 +
 +
'''''Soundtrack Integration...'''''<br>
 +
Integrating the soundtrack with the game was somewhat trivial. We simply sent a GPIO signal from the micro-controller, running the game code, to the micro-controller, interfacing with the mp3 decoder. Once the game began, the soundtrack was initiated and once the game ended, the soundtrack ended.
 +
 +
The mp3 file was loaded onto an SD card. It was then decoded and transferred between the mp3 decoder and the LPC 1758, using an SPI bus. Playing multiple songs proved to be a bit more challenging though and we were not able to solve that particular problem before the demo.
 +
 +
 +
'''''Integrating all of the systems...'''''<br>
 +
Integrating all of our systems required a bit of creativity, due to the amount of wires we ended up with. We used a rectangular box as an enclosure (a makeshift chassis) for all of our systems. We used duct tape and electrical tape, in order to secure the wires, ground plane and power supply to the chassis, in order to prevent movement during gameplay. This was especially important, as player sprite movement depended on feedback from the acceleration sensor and we didn't want to have any connections broken because of moving parts. Also, the USB power supply was quite heavy and had the potential to dislodge other components.
 +
 +
While there was no way to reduce our wiring during the final stages of the project when we developed the enclosure, we did our best to neaten up the varying wiring harnesses throughout the system, in order to avoid a cluttered layout. We used boxcutters, in order to create handles for the enclosure, which also acted as the controller. We also cut holes to route power and signal connectors to the LED matrix, which was mounted on top of the chassis.
 +
 +
Our completed system functioned well during testing, but we experienced some new bugs during the demo, which was unfortunate.
 +
 +
 +
=== <Bug/issue name> ===
 +
'''''Bug encountered during the demo and fixed post demo...'''''<br>
 +
During the demo, we suffered two setbacks at the beginning of each demo. Our first thought was that tangled wires affected the connection between the LPC 1758 and the LED matrix. However after analyzing our code, we found a bug, in the start_game() task, relating to the game_over() task. When the game begins, the start task is supposed to suspend all tasks except for itself and game_over(); This is important because when the starting sequence finishes, the game_over() task blocks for 60 seconds. When the game_over() unblocks, it suspends all in-game tasks except for itself and the the matrix driver task. This triggers the game over message and ends the game. We realized that we had not suspended the game_over() task in the start_game() task, effectively making the game_over() task block as soon as the starting message began. Because the player has to push the trigger button to start the game, the game can stay in the start_game() task indefinitely. However, the time-to-unblock, for the game_over() task will be decrementing while the player has not actually started the game. If the game waits long enough in this state, the game_over() task unblocks and starts running before the game begins. This effectively corrupted our game during the demo.
 +
 +
Our solution: Suspend the game_over() task in the start_game() task. When suspended at the beginning of the task, and resumed at the start of the game, the game_over() task always worked correctly. Its 60 second game timer began counting when the game began, instead of before the game began. This was a simple fix once we recognized it, but it was a critical bug, with catastrophic consequences. We did not catch this bug previously, because we usually started the game right away during the testing phase. Before the demos, we let the LPC 1758 sit in an idle position after running the start_game() task. The game_over() block time was decrementing while we waited and it ended up being called before we started the game during both of our failed attempts. The game at this point runs as expected and the game can be played over and over again, by simply pressing the trigger button during the game_over () task, after every game.
 +
 +
 +
'''''Using the acceleration sensor to control player sprite movement resulted in the sprite flickering...'''''<br>
 +
The player sprite flickered one column to the left or one column to the right intermittently, when moving. When the LPC 1758 was completely still, flickering did not occur.
 +
 +
We tried sampling the acceleration sensor 10 times and averaging the integer result, before scaling it by our chosen step value (83), in order to apply the appropriate bit shift to the player sprite. This did not resolve the flickering.
 +
 +
We also tried processing the acceleration Y-axis feedback in a separate task from where we drew the sprite. We included a 1ms delay for each task, as we were concerned that there may have been overlap of calculating acceleration sensor feedback and applying the bit shift to the player sprite. ''(12/7/18)''
 +
 +
A few modifications to our move_player_sprite() function led to a significant improvement in performance. We increased the acceleration sensor step size (which is used to scale the Y-axis feedback integer value into a bit shift offset) and implemented our software interpretation of bandpass filter. We also stored the difference between the previous acceleration sensor Y-axis feedback value and the current value in a variable. We used the absolute value of this difference to determine if the acceleration sensor had registered significant enough movement to justify changing the current sprite position offset value. We also increased the move_player_sprite() function's block time to 100ms, which allowed it to more accurately track real-time movement (as human hands don't move fast enough to justify a 1ms delay). These modifications significantly reduced player sprite flickering. ''(12/10/18)''
 +
 +
 +
'''''Writing to registers on the MP3 decoder...'''''<br>
 +
 +
When attempting to read the contents of reset values on the VS1053b registers, we were always receiving high impedance. This is the default value for the MISO line in SPI protocol. We changed the master clock speed to 24Mhz and finally we could read the contents of the reset values. Another important modification in our code was to reset the MP3 decoder through the reset pin before writing to the registers. Its important to initialize CS and XDCS pins high because they are active-low configured.
 +
 +
Another issue was that when the system was logging a crash, it was overwriting all the data on the SD card. As a result the mp3 files were corrupted. When attempting to play music, there was no output and I had thought it was a hardware issue. A simple format of the SD card and re-upload of the mp3 after a crash fixed this issue.
  
  

Revision as of 18:37, 18 December 2019

Grading Criteria

  • How well is Software & Hardware Design described?
  • How well can this report be used to reproduce this project?
  • Code Quality
  • Overall Report Quality:
    • Software Block Diagrams
    • Hardware Block Diagrams
      Schematic Quality
    • Quality of technical challenges and solutions adopted.

M&B (Morph & Blend)

Abstract

M&B is a unique and fast game in which the player has to blend and match with the incoming obstacles. The game demands good reflexes to jump and blend in with the approaching obstacles to keep moving forward. The player is required to control his cube-like character that moves to the right, left and forward directions to make long jumps and change color by these obstacles. Instead of the usual jumping over the barriers and pits, the player has to adapt to them by dynamically changing the character’s color before landing on the platform. SJTwo board will be used to implement the game logic, control the RGB matrix and the joysticks/switches. The RGB matrix will be used to display the real-time game statistics such as the player name and their score. The game ends if the player lands on an obstacle of a different color.

Objectives & Introduction

Objective

The objective of this project is to develop a simple, single-player 2D game using LPC174078 microcontroller on an LED matrix display. It focuses on integrating the micro-controller peripheral drivers, led drivers, mp3 player, button controller interface and the application software in FreeRTOS.The button controller interface consists of the button controls for changing color, to control the player's movement and to reset the game. The mp3 player is used to play the game background music.

There are four components in this project:

  • The Display : A 32X64 LED Display Matrix acts as the display of the game.
  • The Controller : The SJ Two Board computes the random obstacle generation and handles the movement and color blend of the player from the button control interface and transfers the information to the display using the GPIO pins.
  • The Button Control Interface : The push buttons on the PCB designed for the game control, interfaced with the game reads the inputs given by the user and relays to the SJ One board to control the player.
  • The MP3 music player : The game background music is played from the mp3 player and relayed to the SJ-Two board using the SPI bus.

Introduction

The Project consists of three main modules:

Button control module: It consists of 7 buttons for the player control; 4 for the different colours to switch, 2 for the directions and 1 for the game reset.

Display Module: It is responsible for controlling 32*64 LED Matrix interfaced to SJ Two Board.

Mp3 application This establishes communication to the display module using SPI.

About the game

  • Player should match the colour of the approaching obstacles to score points by "morphing and blending".
  • Obstacles of random colours are generated randomly.
  • Button control interface enables the user to shift directions and also control the color of the player.
  • The game can be reset at anytime using the reset button.

Team Members & Responsibilities

  • Ryan Will
    • PCB design
    • MP3 implementation
  • Shreeya Mahadevaswamy
    • LED Matrix driver
    • Game Logic- Obstacle
    • Game control configuration
    • Scoreboard
    • Testing
  • Shanmathi Saravanan
    • LED Matrix driver
    • Game Logic- Autorunner
    • Game control configuration
    • Testing
    • Scoreboard

Schedule

Week Date Task Status Completion Date
1 10/10/2019
  • Submission of Project Proposal
  • Completed
  • 10/10/2019
2 10/15/2019
  • Research for Required Components.
  • Submit Schedule and Components List.
  • Completed
  • 10/20/2019
3 10/22/2019
  • Ordering components
  • Familiarize with LED Matrix Datasheet.
  • Completed
  • 11/5/2019
4 10/5/2019
  • Develop Drivers for hardware components.
  • Completed
  • 11/15/2019
5 11/16/2019
  • MP3 implementation
  • Done
  • 12/18/2019
6 11/16/2019
  • 64x32 LED matrix Implementation
  • Completed
  • 11/19/2019
7 11/19/2019
  • Develop Algorithm Design for Game Logic
  • Autorunner and Obstacle generation
  • Completed
  • 11/26/2019
8 11/26/2019
  • Game control configuration
  • Completed
  • 12/17/2019
9 12/3/2019
  • Integration of subsystems
  • Completed
  • 12/17-/2019
10 12/10/2019
  • Final bug fixes and troubleshooting.
  • Completed
  • 12/17/2019
11 12/17/2019
  • Complete wiki report and final demo.
  • In progress
  • 12/18/2019

Bill of Materials (General Parts)

PART NAME

PART MODEL & SOURCE

QUANTITY

COST (USD)

  • SJTwo Boards
Purchased from Preet Kang
2
50.00
  • LED Matrix 32x64
Sparkfun [1]
1
64.99
  • Switches
Amazon [2]
6
7.00
  • DC Barrel Jack Adapter - Female
https://www.amazon.com/Chanzon-Female-Connector-Security-Adapter/dp/B079RCNNCK/ref=sr_1_4?keywords=DC+Barrel+Jack+Adapter+-+Female&qid=1574052181&sr=8-4
1
5.75
  • Power supply
Sparkfun 5V / 4A Power Supply
1
12.95


Design & Implementation

The game design mainly consists of the LED drivers, PCB designs, button control interface for user control, mp3 implementation to play the background music for the game.

Hardware Design

The hardware design of the 32x64 RGB LED matrix panel which is the most important part of the project, uses four data lines namely A,B,C and D which can be addressed and used to control each LED which has following technical specifications:

Parameters:

  • 2048 RGB LEDs
  • 1/16 Scan Rate
  • IDC Connector for Daisy Chaining
  • 5V Supply Voltage


The figure and table below show the pin-out of the RGB LED matrix with description.


Label Name Function
1 R1 High R data
2 G1 High G data
3 B1 High B data
4 R2 Low R data
5 G2 Low G data
6 B2 Low B data
7 A A line selection
8 B B line selection
9 C C line selection
10 D D line selection
11 CLK CLOCK
12 LAT LATCH
13 OE Output Enable
14 GND GND

LED Matrix Control

The LED panel contains 1024 RGB LEDs arranged in a matrix of 32 rows and 64 columns. Each RGB LED contains separate red, green, and blue LED chips assembled together in a single package. The display is subdivided horizontally into two parts, the top half and bottom half consists of 64 columns and 16 rows respectively.

There are different drivers for controlling display’s columns and another set of drivers for controlling rows. To illuminate an LED, the drivers for both the column and the row for that LED must be turned on. To change the color of an LED, the red, green, and blue chips in each LED package are controlled individually and have their own column drivers.

LED Matrix


The panel contains six sets of column drivers; three for the top half of the display and three for the bottom. Each driver has 32 outputs. The three drivers for the top of the display drive the red, green, and blue chips in each of the 64 columns of LEDs in rows 0 to 15 of the panel. The three drivers for the bottom of the display drive the red, green, and blue chips in each of the 64 columns of LEDs in rows 16 to 31 of the panel. The red, green, and blue column drivers for the top half of the display are attached respectively to the R0, G0, and B0 data inputs. The red, green, and blue column drivers for the bottom half of the display are attached respectively to the R1, G1, and B1 data inputs. All six of the 32-bit drivers share common SCLK, LATCH, and BLANK signals.

The display is multiplexed and has a 1/16th duty cycle. This means that no more than one row out of the 16 in the top half of the display and one row out of the 16 in the bottom half of the display are ever illuminated at once. Furthermore, an LED can only be on or off. If both the row and column for an LED are turned on, the LED will be illuminated; otherwise, the LED will be off. To display an image, the entire LED panel must be scanned fast enough so that it appears to display a continuous image without flickering. To display different colors and different brightness levels, the brightness of the red, green, and blue LED chips within each LED package must be adjusted by varying the amount of time that each LED chip is on or off within a single refresh cycle.

Hardware Interface

For the hardware, we designed a PCB board that would be used as a hub to connect the LPC4078, the LED Matrix, and the controls together. The images below show the PCB Board and the PCB schematic that were designed. The PCB features 26 pin connections that can be used to connect the pins of the LPC4078 to the LED Matrix, an MP3 decoder, a controller, or any other components. The PCB board was designed using the EAGLE PCB Design Software.

PCB Board Design

This is a screenshot of the PCB Board that was designed through EAGLE. All of the parts used for the design were plated through holes. This allowed for simple header pins to be soldered directly to the PCB rather than using surface mounted parts.

PCB Board

PCB Schematic Design

This is a screenshot of the PCB Board Schematic that was designed through EAGLE and used to build design the PCB Board. The schematic features a large number of header pins that can be used to connect the LPC4078 to a variety of components, including an LED matrix, a button controller, and an MP3 Decoder. Several pins were also added to supply both a VCC and a GND to all of the components used in the design.

PCB Schematic

Software Design

The main code logic blocks which are the random obstacle generation, obstacle motion enabling and obstacle color mismatch detection blocks are as follows: OBSTACLE GENERATION:

bool generate_obstacle_rand(void) {

 bool ret;
 uint8_t randomize_color = (rand() % 4) + 1; // generating random colour
 rand_row = (rand() % 17) + 9;  // generating a random row for obstacle generation
 while (!(((row_old - rand_row) >= obstacle_gap) || ((rand_row - row_old) >= obstacle_gap))) {  // loop  is used to generate a new row number in comparison with the last generated row number
   rand_row = (rand() % 17) + 9;
 }
 while (colour_old == randomize_color || ((colour_old == yellow1) && (randomize_color == cyan1))) { 
   randomize_color = (rand() % 3) + 1;
 }
 if (randomize_color == cyan1) {
   randomize_color = yellow1;
 }
 int8_t row = rand_row;
 for (int8_t col = 63; col > 52; col--) {
     drawPixel(row, col, randomize_color);
     drawPixel(row + 1, col, randomize_color);
     ret = 0;
 }
 row_old = rand_row;
 colour_old = randomize_color;
 ret = 1;
 return (ret);

}

OBSTACLE MOTION LOGIC:

void shift_obstacle_left(void) {

 int8_t temp;
 for (int8_t row1 = 9; row1 < 28; row1++) {
   for (int8_t col1 = 0; col1 < 63; col1++) {
     ledmatrix_buffer[row1][col1] = ledmatrix_buffer[row1][col1 + 1]; // move all element to the left except first
   }
   ledmatrix_buffer[row1][63] = 0;
 }

}

OBSTACLE BLEND COLOR MISMATCH DETECTION:

bool detect_collision(void) {

 bool detect = 0;
 if (((ledmatrix_buffer[autorunner_row][4] != ledmatrix_buffer[autorunner_row][5]) &&
      (ledmatrix_buffer[autorunner_row][5] != 0)) ||
     ((ledmatrix_buffer[autorunner_row + 1][5] != ledmatrix_buffer[autorunner_row + 1][6]) &&
      (ledmatrix_buffer[autorunner_row + 1][6] != 0)) ||
     ((ledmatrix_buffer[autorunner_row + 2][4] != ledmatrix_buffer[autorunner_row + 2][5]) &&
      (ledmatrix_buffer[autorunner_row + 2][5] != 0)) ||
     ((ledmatrix_buffer[autorunner_row + 3][5] != ledmatrix_buffer[autorunner_row + 3][6]) &&
      (ledmatrix_buffer[autorunner_row + 3][6] != 0))) {
   detect = 1;
 } else {
   detect = 0;
 }
 return (detect);

}

MP3 Player Tasks:

void MP3_Play_Task(void *pvParameters) {

       BYTE decoder_buffer[512];
       mp3_decoder_init();
       while(1)
       {
               if(xQueueReceive(MP3Queue, decoder_buffer, 1000))
               {
                       //printf("track playing");
                       play_mp3_track(decoder_buffer);
               }
       }

}

void Read_MP3_Task(void *pvParameters) {

       while(1)
       {
               readMP3File();
               vTaskDelay(1000);
       }

}

MP3 Decoder Driver:

void mp3_decoder_init() {

   ssp__init(24);
   gpio_x__set_as_output(1, 28); //Set pin 1.28 as output for RESET signal
   gpio_x__set_as_output(1, 20); //Set pin 1.20 as output for DCS signal
   gpio_x__set_as_output(1, 31); //Set pin 1.31 as output for CS signal
   //gpio_x__set(0, 25, true); //Set pin 0.25 high initially
   //gpio_x__set(0, 26, true); //Set pin 0.26 high initially
   //gpio_x__set(1, 30, true); //Set pin 1.30 high initially
   gpio_x__set_high(1, 28);
   gpio_x__set_high(1, 20);
   gpio_x__set_high(1, 31);
   gpio_x__set_as_input(0, 26); //Set pin 0.26 as input for DREQ
   write_to_decoder(SCI_MODE, 0x0800);         //Set up MODE register
   //write_to_decoder();                         //Set up Bass/Treble register
   write_to_decoder(SCI_CLOCKF, 0x2000);       //Set up Clock register
   write_to_decoder(SCI_AUDATA, 0xAC45);       //Set up AudioData register               `
   write_to_decoder(SCI_VOL, 0x2424);          //Set up Volume register
   uint16_t data1 = read_from_decoder(SCI_MODE);
   uint16_t data2 = read_from_decoder(SCI_CLOCKF);
   uint16_t data3 = read_from_decoder(SCI_AUDATA);
   uint16_t data4 = read_from_decoder(SCI_VOL);
   printf("SCI_MODE = %x\n", data1);
   printf("SCI_CLOCKF = %x\n", data2);
   printf("SCI_AUDATA = %x\n", data3);
   printf("SCI_VOL = %x\n", data4);
   //u0_dbg_printf("mode data = %x\n", data);

}

void play_mp3_track(uint8_t buffer[]) {

   for(uint8_t byteIndex = 0; byteIndex < 16; byteIndex++)
   {
       while(!(gpio_x__get_level(0, 26)))
       {
           //vTaskDelay(1);
       }
       gpio_x__set(1, 20, false);
       //gpio_x__set_low(0, 26);
       //gpio_x__set_low(1, 30);
       for(uint16_t i = 32 * byteIndex; i < (32 * byteIndex) + 32; i++)
       {
           ssp__exchange_byte(buffer[i]);
       }
       while(!(gpio_x__get_level(0, 26)))
       {
           //vTaskDelay(1);
       }
       gpio_x__set(1, 20, true);
       //gpio_x__set_high(0, 26);
       //gpio_x__set_high(1, 30);
   }

}

Implementation

This section includes implementation, but again, not the details, just the high level. For example, you can list the steps it takes to communicate over a sensor, or the steps needed to write a page of memory onto SPI Flash. You can include sub-sections for each of your component implementations.

In order to play music from the MP3 decoder, the MP3 file needs to be pulled from an SD card and into a buffer, and the data needs to be fed to the MP3 Decoder through the SPI Bus.

Testing & Technical Challenges

Developing a driver for the LED matrix...

The biggest technical challenge that we faced was interfacing the LPC 1758 micro-controller, with the Adafruit 32 x 64 LED matrix.

Adafruit (the 32 x 64 LED matrix's manufacturer) provides a basic tutorial on their website, in order to help with interfacing the LED matrix to a micro-controller. The tutorial briefly describes the functions of the pins on the board's input and output connector ports. It also describes how to connect jumper wires to the ribbon cable (which comes packaged with the board), which connects to the matrix's input and output connector ports. It assumes that the user will utilize an Arduino-type micro-controller to drive the matrix's RGB LED's.

Because we used the LPC 1758 to drive the LED matrix, our choices were to either adapt a third party driver library designed for an Arduino, or develop a new driver. We chose the latter option, as the third party Arduino-focused API's were difficult to fully understand. We also felt that developing our own driver would allow us to gain a strong understanding of all aspects of micro-controller to LED matrix integration. This proved useful both for firmware development and debugging.

The first part of developing the LED matrix driver, was learning how each of the 16 input pins affected the matrix display.


The LED matrix's input socket looked as follows:

Adafruit 32x32 LED Matrix socket.png


GND...
The 3 GND pins were the easiest to understand. They provided a ground signal reference for the matrix's RGB LED's. The GND pins on the LED matrix were connected to GND on the LPC 1758, through a ground plane on the PCB, allowing the micro-controller and LED matrix to share the same signal reference.


A B C D...
Pins A, B, C and D, were used as control bits for the LED matrix's row multiplexer. The 32 x 64 matrix is made up of two individual 16 x 64 LED panels. Each panel contains 16 rows and 32 columns. The binary values of the multiplexer signals, A, B, C and D, are used to determine which row of LED's is being driven. Each panel drives the same row of LED's at a time, based on these values. For example, if A, B, C and D are set to 0000, then the LED's in the first row of each panel can be driven. If A, B, C and D are set to 0001, then the LED's in the second row of each panel can be driven. This pattern continues until A, B, C and D are set to 1111, which corresponds to sixteenth (bottom) row of each panel. We set our row iterating value, as i = (i + 1) % 16, so that A, B, C and D, would be set to 0000, after finishing latching data into the sixteenth row. This allowed the multiplexer to continuously loop through each row in each panel, from top to bottom. We applied a delay of 1ms after each row iteration, thereby setting the overall scan rate to 62Hz. This scan rate was too fast for the human eye to detect, which allowed the LED matrix to display objects steadily, without annoying flickering compromising the image quality.


R1 B1 G1
R2 G2 B2...
Pins R1, B1, G1 and R2, B2, G2, were the 6 control signals used to drive the matrix's RGB LEDs, different combinations of red, blue and green (depending on their binary values). R1, B1 and G1, controlled indiidual LED colors on the top panel (panel 1), while R2, B2 and G2, controlled individual LED colors on the bottom panel (panel 2). The way these color combinations manifested on the board, were as follows:

R B G COLOR
0 0 0 OFF
0 0 1 GREEN
0 1 0 BLUE
0 1 1 CYAN
1 0 0 RED
1 0 1 YELLOW
1 1 0 PURPLE
1 1 1 WHITE

In order to understand how these signals drive individual LEDs on the matrix, one must understand the hardware that they control. The matrix's RGB LEDs are driven by 6 column drivers, each having 32 outputs. Each panel has 3 column drivers. Each column driver can drive 32 LED's in each panel, either red, green or blue (because each of the 3 column drivers controls one of these 3 colors). As a result, the 6 column drivers can drive up to 192 individual red, green and blue LED's (each RGB LED consists of 3 individual red, green and blue LED's) at a time.


CLK LAT OE...
Each column driver is composed of a serial data input, a blanking input, a shift register, and a parallel output register. Data bits are passed into the shift register on the rising edge of each clock cycle. The clock is controlled using the signal CLK. Because there are 32 RGB LED's per row, 32 clock cycles must occur, in order to drive all LED's in a row. After 32 bits of data are passed into the shift register, the LATCH signal (which is set low before shifting in data bits to the shift register) must be asserted, in order to transfer the data bits from the shift register to the parallel output register. Output enable (OE) is pulled low at this point, in order to enable the column driver. It is pulled high again, when transitioning to a new row.

CMPE244 F18 T4 Rgb-led-panel-display-organization.png CMPE244 F18 T4 Rgb-led-panel-shift-register.png

We designed our driver to operate on 32 bit, bit strings (of type uint32_t), in order to utilize all 32 LEDs in each row of the matrix. Each 32 bit string, was stored in a 32 element array of type, uint32_t. We assigned the base memory address of the array to a pointer, and used integer offset values to pass the different bit strings, stored in the array, into the matrix driver function. This method allowed us to pass 32 unique bit strings into all 32 rows of the LED matrix. The integer offsets corresponded to row numbers. For example an offset of 4, would increment the pointer, pointing at the memory address of row 0, by 4 memory addresses. Dereferencing the pointer with this offset (4), would give access to data bits governing the output of row 4 of the LED matrix (the fifth row).

For each row, we used a for loop (which iterated from 0 to 31) in order to shift all 192 data bits into the shift registers. We latched the data bits into the parallel output register, after shifting was completed, which allowed us to illuminate different individual LED's in each row. This allowed us to use 32 bit, bit strings, to create data structures and graphics.

The column driver also allowed us to give the illusion of motion, by turning different LED's on and off in adjacent rows.


Integrating an acceleration sensor to move an avatar...

Another significant technical challenge that we faced, was integrating the LPC 1758's acceleration sensor with a virtual avatar (the player sprite). We used feedback from the acceleration sensor's Y-axis tilt output, to govern the player sprite's lateral movement. In this day, the play had one dimension of control over the sprite, by adjusting the Y-axis tilt of the LPC 1758.

While the task using acceleration sensor feedback to control movement was fairly trivial, it induced a bug in the game. The player sprite flickered one column to the left or one column to the right intermittently, when moving. When the LPC 1758 was completely still, flickering did not occur.

We tried sampling the acceleration sensor 10 times and averaging the integer result, before scaling it by our chosen step value (83), in order to apply the appropriate bit shift to the player sprite. This did not resolve the flickering.

We also tried processing the acceleration Y-axis feedback in a separate task from where we drew the sprite. We included a 1ms delay for each task, as we were concerned that there may have been overlap of calculating acceleration sensor feedback and applying the bit shift to the player sprite.

A few modifications to our move_player_sprite() function led to a significant improvement in performance. We increased the acceleration sensor step size (which is used to scale the Y-axis feedback integer value into a bit shift offset) and implemented our software interpretation of bandpass filter. We also stored the difference between the previous acceleration sensor Y-axis feedback value and the current value in a variable. We used the absolute value of this difference to determine if the acceleration sensor had registered significant enough movement to justify changing the current sprite position offset value. We also increased the move_player_sprite() function's block time to 100ms, which allowed it to more accurately track real-time movement (as human hands don't move fast enough to justify a 1ms delay). These modifications significantly reduced player sprite flickering.


Tracking a moving sprite and integrating shooting...
Tracking the player sprite was a fairly trivial task, due to our player sprite only moving in one dimension (left and right). Because the player sprite was holding a 1 pixel wide gun, we continuously scanned the row where the gun was located, in order to determine which column of the LED matrix would be set as the projectile path. We constantly scanned for a data bit set to 1, which when found, would let us know exactly where in the row that the gun was located.

A "shoot" signal was triggered by a push button sensor, which unblocked the "fire_a_shot()" function. This function passed the gun's column location to the "projectile_path()" function, allowing the projectile sprite to travel vertically, towards a moving enemy sprite (or nothing at all).


Enemy sprites and re-spawn time...
We created 5 unique enemy sprites, using combinations of adjacent bit strings. Each enemy sprite traversed the LED matrix laterally and was managed by its own task. We used a random number generator in addition with a fixed-time modulus, in order to implement random re-spawn times of sprites. The enemy sprite's position in the virtual LED matrix array was incremented by one column, after a set time delay. Each enemy sprite had a unique time delay, in order to allow some sprites to move faster than others and thus be harder to shoot. Faster sprites were worth more points (when shot), while slower sprites, while easier to hit, resulted in less points added to the player's total game score.


Hit detection and scorekeeping...
Hit detection proved less challenging that we initially thought that it would be. When a projectile was fired at an enemy sprite, the projectile's position was tracked as it traversed the LED matrix, vertically. When the projectile's position was within one pixel of the enemy sprite (and directly beneath it), a hit was registered. Upon a hit, the all data bits in the rows containing the vanquished enemy sprite were cleared to "0", This effectively removed the enemy sprite from the game, until a new identical enemy sprite re-spawned. We incorporated a random number driven delay, to control the re-spawn time of enemy sprites. This random re-spawn time was applied upon hits, as well as if an enemy sprite successfully traversed all 32 columns of the matrix (without being hit by a projectile).


Start Procedure...
In order to create a "Start" message for the game (essentially an introduction to the game), we created a task function called "Start_Game()". This task loaded data bits comprising a starting message, into the virtual LED matrix array. It also used the unique task handles of the in-game task functions, in order to suspend them, while it ran. After a fixed delay of 10 seconds, All data bits in the virtual LED matrix were set to 0; creating a blank screen. Start_Game() then used the task handles of each in-game task, to resume them. This commenced the actual Spartan Warrior game. We also sent a high (1) signal to the other LPC 1758 micro-controller in the system, which interfaced with the mp3 decoder. Upon receiving this signal, the other micro-controller worked with the mp3 decoder to play the soundtrack.


Game Over Procedure...
In order to implement the "Game Over" message, we began by setting a delay of 60 seconds. This allowed the task to block for one minute (or the length of the actual in-game time). We adjusted the input integer parameter of the delay_ms() function, in order to ensure that 60 seconds of actual game time was allowed, before this task was called. We chose this value for the game time, for demo purposes only. We could have adjusted it to any value we wished. Once the delay time expired, all in-game tasks were suspended and all data bits in the virtual LED matrix array, were set to 0. The message "Game Over", was loaded into the array and displayed on the screen, as well as the player's score total (on the lower panel).


MP3 Decoder... The system uses two LPC1758 boards; one driving the LED matrix display and one, which reads mp3 data from an SD card using SPI. The two boards communicate with each other using a GPIO pin. A binary signal (high) indicates when to start the soundtrack. It is sent during the start_screen() task. The game_over() task pulls the signal low when it unblocks, signalling the LPC 1758, playing the soundtrack, to stop outputting the audio signal.

An audio decoder was used to collect the data from the queue, translate it and play it on the speaker, connected at the output. When the game reaches the end, the GPIO pin is reset which signals the decoder to stop retrieving data from the queue. The code uses the following to get the data from the SD Card and insert it in the queue:

[[Storage::read("1:track1.mp3", &dataChunk[0], 4096, songoffset); 
songoffset is a data member that stores the starting address of 4k bytes to send to the queue.]]


Soundtrack Integration...
Integrating the soundtrack with the game was somewhat trivial. We simply sent a GPIO signal from the micro-controller, running the game code, to the micro-controller, interfacing with the mp3 decoder. Once the game began, the soundtrack was initiated and once the game ended, the soundtrack ended.

The mp3 file was loaded onto an SD card. It was then decoded and transferred between the mp3 decoder and the LPC 1758, using an SPI bus. Playing multiple songs proved to be a bit more challenging though and we were not able to solve that particular problem before the demo.


Integrating all of the systems...
Integrating all of our systems required a bit of creativity, due to the amount of wires we ended up with. We used a rectangular box as an enclosure (a makeshift chassis) for all of our systems. We used duct tape and electrical tape, in order to secure the wires, ground plane and power supply to the chassis, in order to prevent movement during gameplay. This was especially important, as player sprite movement depended on feedback from the acceleration sensor and we didn't want to have any connections broken because of moving parts. Also, the USB power supply was quite heavy and had the potential to dislodge other components.

While there was no way to reduce our wiring during the final stages of the project when we developed the enclosure, we did our best to neaten up the varying wiring harnesses throughout the system, in order to avoid a cluttered layout. We used boxcutters, in order to create handles for the enclosure, which also acted as the controller. We also cut holes to route power and signal connectors to the LED matrix, which was mounted on top of the chassis.

Our completed system functioned well during testing, but we experienced some new bugs during the demo, which was unfortunate.


<Bug/issue name>

Bug encountered during the demo and fixed post demo...
During the demo, we suffered two setbacks at the beginning of each demo. Our first thought was that tangled wires affected the connection between the LPC 1758 and the LED matrix. However after analyzing our code, we found a bug, in the start_game() task, relating to the game_over() task. When the game begins, the start task is supposed to suspend all tasks except for itself and game_over(); This is important because when the starting sequence finishes, the game_over() task blocks for 60 seconds. When the game_over() unblocks, it suspends all in-game tasks except for itself and the the matrix driver task. This triggers the game over message and ends the game. We realized that we had not suspended the game_over() task in the start_game() task, effectively making the game_over() task block as soon as the starting message began. Because the player has to push the trigger button to start the game, the game can stay in the start_game() task indefinitely. However, the time-to-unblock, for the game_over() task will be decrementing while the player has not actually started the game. If the game waits long enough in this state, the game_over() task unblocks and starts running before the game begins. This effectively corrupted our game during the demo.

Our solution: Suspend the game_over() task in the start_game() task. When suspended at the beginning of the task, and resumed at the start of the game, the game_over() task always worked correctly. Its 60 second game timer began counting when the game began, instead of before the game began. This was a simple fix once we recognized it, but it was a critical bug, with catastrophic consequences. We did not catch this bug previously, because we usually started the game right away during the testing phase. Before the demos, we let the LPC 1758 sit in an idle position after running the start_game() task. The game_over() block time was decrementing while we waited and it ended up being called before we started the game during both of our failed attempts. The game at this point runs as expected and the game can be played over and over again, by simply pressing the trigger button during the game_over () task, after every game.


Using the acceleration sensor to control player sprite movement resulted in the sprite flickering...
The player sprite flickered one column to the left or one column to the right intermittently, when moving. When the LPC 1758 was completely still, flickering did not occur.

We tried sampling the acceleration sensor 10 times and averaging the integer result, before scaling it by our chosen step value (83), in order to apply the appropriate bit shift to the player sprite. This did not resolve the flickering.

We also tried processing the acceleration Y-axis feedback in a separate task from where we drew the sprite. We included a 1ms delay for each task, as we were concerned that there may have been overlap of calculating acceleration sensor feedback and applying the bit shift to the player sprite. (12/7/18)

A few modifications to our move_player_sprite() function led to a significant improvement in performance. We increased the acceleration sensor step size (which is used to scale the Y-axis feedback integer value into a bit shift offset) and implemented our software interpretation of bandpass filter. We also stored the difference between the previous acceleration sensor Y-axis feedback value and the current value in a variable. We used the absolute value of this difference to determine if the acceleration sensor had registered significant enough movement to justify changing the current sprite position offset value. We also increased the move_player_sprite() function's block time to 100ms, which allowed it to more accurately track real-time movement (as human hands don't move fast enough to justify a 1ms delay). These modifications significantly reduced player sprite flickering. (12/10/18)


Writing to registers on the MP3 decoder...

When attempting to read the contents of reset values on the VS1053b registers, we were always receiving high impedance. This is the default value for the MISO line in SPI protocol. We changed the master clock speed to 24Mhz and finally we could read the contents of the reset values. Another important modification in our code was to reset the MP3 decoder through the reset pin before writing to the registers. Its important to initialize CS and XDCS pins high because they are active-low configured.

Another issue was that when the system was logging a crash, it was overwriting all the data on the SD card. As a result the mp3 files were corrupted. When attempting to play music, there was no output and I had thought it was a hardware issue. A simple format of the SD card and re-upload of the mp3 after a crash fixed this issue.


<Bug/issue name>

Discuss the issue and resolution.

Conclusion

Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

Project Video

Upload a video of your project and post the link here.

Project Source Code

References

Acknowledgement

Any acknowledgement that you may wish to provide can be included here.

References Used

List any references used in project.

Appendix

You can list the references you used.