F21: FollowMe
Contents
Abstract
FollowMe is a video game where you follow the correct directions. The LED matrix will display a series of arrows and the player or players need to correctly wave their hand over their gesture sensor to earn points. However, watch out for the background color of the arrow. If the background is green, then the players need to wave their hand in that direction; i.e, an arrow pointing left with a green background means wave your hand from the right to the left (wave left). If the background is red, then the players need to wave their hand in the opposite direction; i.e, an arrow pointing up with a red background means wave your hand from the top to the bottom (wave down).
There'll be two modes:
1. Timed mode: each player tries to get the most correct waves until the time runs out. The rate at which the arrows change will increase as the timer counts down.
2. Endurance mode: each player tries to stay alive the longest. One wrong wave then you're eliminated, the last player standing wins.
At the end, the LED matrix will display the leaderboard for the players and how many waves/instructions they got correct.
- Block Diagram of Follow Me needs to be added
Objectives & Introduction
Project Introduction
1.Analyse hand waving/movement using a gesture sensor
2.Connect different microcontrollers wirelessly to exchange information
3.Read a file from SD card on the microcontroller and get the MP3 data using MP3 decoder
4.Display different screens according to the game level(Title screen, Arrows and leaderboards)
Project Objectives
1. Gestor Control: APDS-9960 gesture sensor on SJ2 board should respond to hand waving. I2C interface will be used for this sensor.
2. ZIGBEE wireless: UART interface used to connect the wireless devices, which will send information to other boards.
3. MP3 decoder: MP3 data will be fetched from mini SD card reader using SPI interface.
4. 64x64 RGB LED matrix: LED will be controlled by GPIO of SJ2 board. The LED will display game graphics.
5. 3D Printed enclosure: The system will be enclosed in a 3D printed enclosure.
Team Objectives
1.Learn and understand the APIs of the Real Time Operating System
2.Use different embedded protocol to interface the system
3.Learn to design a module and interface the modules by reading datasheets
4.Integration and testing to deliver the final product
5.Track and respond to the reported bug and document the bugs and solutions.
Team Members & Technical Responsibilities
- Jonathan Doctolero
- Gesture Sensor Driver
- Gameplay Development
- Enclosure Design
- Priyanka Rai
- RGB Matrix Driver
- Wireless Driver/Application
- Ravi Kiran Dendukuri
- PCB Design
- MP3 Decoder Design
- Yashwanth Kumar Nelakuditi
- PCB Design
- MP3 Decoder Design
Administrative Responsibilities
| Administrative Roles | ||||
|---|---|---|---|---|
|
Jonathan Doctolero | |||
|
Priyanka Rai | |||
|
Ravi Kiran Dendukuri | |||
|
Yashwanth Kumar Nelakuditi | |||
Schedule
| Week# | Start Date | End Date | Task | Status | Deliverable |
|---|---|---|---|---|---|
| 1 |
|
|
|
|
|
| 2 |
|
|
|
|
|
| 3 |
|
|
|
|
|
| 4 |
|
|
|
|
|
| 5 |
|
|
|
|
|
| 6 |
|
|
|
|
|
| 7 |
|
|
|
|
|
| 8 |
|
|
|
|
|
| 9 |
|
|
|
|
|
| 10 |
|
|
|
||
| 11 |
|
|
|
Bill of Materials (General Parts)
| PART NAME |
PART MODEL & SOURCE |
QUANTITY |
COST PER UNIT (USD) |
|---|---|---|---|
|
|
|
|
|
|
| |
|
|
| |
|
|
|
|
|
|
|
|
|
|
| |
|
|
|
|
Printed Circuit Board
Design & Implementation
The design section can go over your hardware and software design. Organize this section using sub-sections that go over your design and implementation.
RGB LED Matrix
A 64 x 64 RGB LED Matrix will be powered up through a 5V/4A DC adapter and is interfaced with the board to play the game 'FOLLOW ME' with the background music. The matrix has 2 planes (upper and lower), both of which will be programmed separately. Planes are made by dividing 64 rows into 2 halves, i.e. first 32 rows in plane 1 and the remaining 32 rows in plane 2. Five signals (viz. A, B, C, D, and E) which are connected to 5 GPIOs of the SJ-2 board are used to select rows from each plane. R1, G1, B1 signals with 64-bit registers are used to address individual led from the selected row in plane 1. Similarly, R2, G2, B2 signals used to address individual led from the selected row in plane 2. A single clock is interfaced to these 6 64bit shift registers. Hence, at one clock signal, we fill and enable a column corresponding to two selected rows. Once the clocking and register shifting is done, the data need to be latched to the register which in turn enables the corresponding LED. Then the LATCH is set to mark end of the row and reset to move to next row. The output enable signal is enabled to push the data to the selected row. The output enable signal is disabled again to select the other row. All of these steps are repeated at very less time intervals so that the human eye perceives it as one complete frame (Persistence of Vision).
Pin Configuration
| Pin | Pin Description |
|---|---|
| R1 | Sets upper panel's Red data |
| G1 | Sets upper panel's Green data |
| B1 | Sets upper panel's Blue data |
| R2 | Sets lower panel's Red data |
| G2 | Sets lower panel's Green data |
| B2 | Sets lower panel's Blue data |
| A | Sets row bit 0 |
| B | Sets row bit 1 |
| C | Sets row bit 2 |
| D | Sets row bit 3 |
| E | Sets row bit 0 |
| nOE | Set to switch the LEDs off when transitioning from one row to the next |
| LATCH | Set to mark completion of one row |
| CLK | Set to access each pixel |
| GND | Ground pins to be connected with board's GND. |
Below are the technical specifications:
| Spec | Value |
|---|---|
| Pitch | 4mm |
| Resolution | 32 x 64 = 2048 dots |
| Panel dimensions (l x b x h) in mm | 256 x 128 x 13 |
| Working voltage/current rating | 5v / 4A (max) |
| Scan Rate | 1 / 16 |
| Pixel component configuration (R,G,B) | (1,1,1) |
| Weight | 0.24 kg |
Hardware Design
The hardware involves 5V/4A DC power supply adapter, barrel jack connectors and IDC cables to power up the LED matrix and communicate with the board. Below is the pin interfacing diagram.
Software Design
1. Fruit Fury - Game mode
There are 2 tasks involved to ensure functioning of the game.
a) Task 1:
- Display the game's title screen. On pressing the onboard switch (SW1), game begins.
- Also used to display game over screen / win screen.
b) Task 2:
- Enter into game mode to play the game.
- Read accelerometer values (board orientation) and render game frame.
- Detect the player's board movement, compare it with fruit's location.
- Display score, lives and smash effects.
- Press another switch (SW0) to stop game and return to title screen or to restart the game once it is over.
Game rules:
a) Game comprises of 3 levels (level 2 on reaching score 15 and level 3 on reaching score 30) and 5 lives in total.
b) Primary objective is to tilt the board to the direction (North West, North East, South West or South East) in which the fruit appears in order to smash it.
c) Score increases by 1 point for smashing each fruit.
d) Bombs start appearing after completion of level 1, which costs a life when hit.
e) Level 3 is faster so make sure you tune your reflexes up.
f) Reach score 50 to win the game.
Bonus:
a) Streaks are rewarding. Smash 10 fruits straight up to win an extra life.
b) Look out for star fruits. Hit them to gain +2 points.
Choose your song to take all the inspiration you can and start smashing!
2. Music mode
a) Press the button to enjoy music-only mode. Graceful audio spectrum bands are displayed that dance to the tune of the chosen song.
b) A display is prompted to the user whenever music is paused or changed to next or previous song.
Implementation
LED Driver
1. Initialize the LED matrix by configuring necessary pin directions.
2. Disable Output Enable (OE) GPIO before feeding matrix data.
3. Select the required row by setting bits on A, B, C, D GPIO pins.
4. Loop through the pixels (columns) in the selected row and set the pixel color on R, G, B GPIO pins.
5. Set zero on R, G, B GPIO pins to mask that particular pixel.
6. Set and Reset the clock for pushing the R, G, B bits for each column.
7. Issue latch to mark the row's completion and reset latch before going to next row.
8. Follow the steps 2 to 7 for other rows.
for (uint8_t row = 0; row < (MAX_ROW / 2); row++) {
disableOE();
set_row(row);
for (uint8_t col = 0; col < MAX_COL; col++) {
LPC_GPIO0->PIN |= (1 << CLK);
set_color_bottom(game_matrix[row + (MAX_ROW / 2)][col]);
set_color_top(game_matrix[row][col]);
LPC_GPIO0->PIN &= ~(1 << CLK);
}
LPC_GPIO0->PIN |= (1 << LAT);
LPC_GPIO0->PIN &= ~(1 << LAT);
enableOE();
vTaskDelay(1);
}
Fruit Fury - Game mode
1. Generate the fruit's initial coordinate using random function (used srand() with time value as seed to get random pattern sets).
2. Construct the LED matrix array by drawing the fruit and borders.
3. Push the entire matrix data to LED driver to display the constructed matrix frame.
4. Then read the player's orientation and match with fruit's direction.
5. Modify game parameters (score, lives and level) based on player's results.
6. Stop game if all lives are over and display game over screen by switching to game init task (Task 1).
/* Push button to exit game */
if (LPC_GPIO0->PIN & (1 << SW0))
is_start_game = false;
if (is_start_game == true) {
/* Generate fruit's initial coordinate position */
get_fruit_begin_coord(&row_pt, &col_pt, &quadrant);
/* Generate final frame by considering fruit object at that position */
construct_game_matrix(row_pt, col_pt);
/* Draw final frame matrix of the game */
draw_final_game_frame();
/* Get the board direction from the user and increment score count */
compute_game_params(row_pt, col_pt, quadrant);
} else
vTaskResume(display_game_init_screen_t);Spectrum display - Music mode
1. Read the audio frequency values from the graphic equalizer task and store it as an array.
2. Based on value of each frequency band, assign a height of the band (in pixels) to be drawn on the LED matrix.
3. Set a unique color for each band.
3. Push the entire matrix data to LED driver to display the constructed matrix frame.
4. Repeat steps 1 to 3 at required intervals to display repeating bands.
for (row = MAX_ROW - 1; row >= 1; row--) {
for (col = 7; col <= 62; col++) {
if ((row >= (MAX_ROW - band_height[0])) && col >= 7 && col <= 13 &&
band_num <= 0) {
band_matrix[row][col] = RED;
}
if ((row >= (MAX_ROW - band_height[1])) && col >= 14 && col <= 20 &&
band_num <= 1) {
band_matrix[row][col] = GREEN;
}
if ((row >= (MAX_ROW - band_height[2])) && col >= 21 && col <= 27 &&
band_num <= 2) {
band_matrix[row][col] = YELLOW;
}
if ((row >= (MAX_ROW - band_height[3])) && col >= 28 && col <= 34 &&
band_num <= 3) {
band_matrix[row][col] = BLUE;
}
if ((row >= (MAX_ROW - band_height[4])) && col >= 35 && col <= 41 &&
band_num <= 4) {
band_matrix[row][col] = PURPLE;
}
if ((row >= (MAX_ROW - band_height[5])) && col >= 42 && col <= 48 &&
band_num <= 5) {
band_matrix[row][col] = CYAN;
}
if ((row >= (MAX_ROW - band_height[6])) && col >= 49 && col <= 55 &&
band_num <= 6) {
band_matrix[row][col] = WHITE;
}
}
}Software Design
Show your software design. For example, if you are designing an MP3 Player, show the tasks that you are using, and what they are doing at a high level. Do not show the details of the code. For example, do not show exact code, but you may show psuedocode and fragments of code. Keep in mind that you are showing DESIGN of your software, not the inner workings of it.
Implementation
This section includes implementation, but again, not the details, just the high level. For example, you can list the steps it takes to communicate over a sensor, or the steps needed to write a page of memory onto SPI Flash. You can include sub-sections for each of your component implementations.
Testing & Technical Challenges
Describe the challenges of your project. What advice would you give yourself or someone else if your project can be started from scratch again? Make a smooth transition to the testing section and describe what it took to test your project.
Include sub-sections that list out a problem and solution, such as:
<Bug/issue name>
Discuss the issue and resolution.
LED Matrix: faulty matrix , needed to buy another one. Lessons: order the obvious component as soon as possible and test it.
- Measure the resistance between 5v and ground terminal
Conclusion
Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?
Project Video
Upload a video of your project and post the link here.
Project Source Code
References
Acknowledgement
Any acknowledgment that you may wish to provide can be included here.
References Used
List any references used in project. https://www.prf.jcu.cz/generator-led-matrix/index.htm
Appendix
You can list the references you used.
