Difference between revisions of "F21: FollowMe"

From Embedded Systems Learning Academy
Jump to: navigation, search
(Team Members & Technical Responsibilities)
(Project Video)
 
(208 intermediate revisions by the same user not shown)
Line 1: Line 1:
=== Grading Criteria ===
+
== Abstract ==
<font color="green">
+
FollowMe is a video game where you follow the correct directions. The LED matrix will display a series of arrows and the player needs to correctly wave their hand over their gesture sensor to earn points. However, watch out for the background color of the arrow. If the background is green, then the players need to wave their hand in that direction; i.e, an arrow pointing left with a green background means wave your hand from the right to the left (wave left). If the background is red, then the players need to wave their hand in the opposite direction; i.e, an arrow pointing up with a red background means wave your hand from the top to the bottom (wave down).
*  How well is Software & Hardware Design described?
 
*  How well can this report be used to reproduce this project?
 
*  Code Quality
 
*  Overall Report Quality:
 
**  Software Block Diagrams
 
**  Hardware Block Diagrams
 
**:  Schematic Quality
 
**  Quality of technical challenges and solutions adopted.
 
</font>
 
  
== '''Project Title''' ==
+
The game has a timer mode. The player tries to get the most correct waves until the time runs out and the arrows change every five seconds.
Parking Frenzy
 
  
== '''Abstract''' ==
+
== Block Diagrams ==
Parking Frenzy is car parking video game. The player will be able to move the car in right or left direction, the road itself will be descending towards the car and will also consist of obstacles such as other vehicles. The road will have lanes through which the car must navigate to ultimately be able to slot itself into the right parking spot along side the road. The parking spots will be randomly generated along the edge of the LED matrix and the player must navigate the car into the spot at the right time or else it will result in a crash. The difficulty of the game will increase as the player reaches a certain amount of points. The difficulty will increase in terms of the speed of the car and the number of obstacles. The final score of the player will be displayed at the end of every game. a speaker attached to the board will generate the game sounds.
+
[[File:MainScreen.png |200px]] [[File:GameMode.png|300px]]
  
== '''Objectives & Introduction''' ==
+
[[File: PlayerInput.png|200px]] [[File: PlayerGame.png|150px]]
Show list of your objectives. This section includes the high level details of your project.  You can write about the various sensors or peripherals you used to get your project completed.
 
  
== '''Team Members & Technical Responsibilities''' ==
+
== Objectives & Introduction ==
*  Jonathan Doctolero
+
'''Project Introduction'''
 +
 
 +
'''1.'''Analyse hand waving/movement using a gesture sensor
 +
 
 +
'''2.'''Connect different microcontrollers wirelessly to exchange information
 +
 
 +
'''3.'''Read a file from SD card on the microcontroller and get the MP3 data using MP3 decoder
 +
 
 +
'''4.'''Display different screens according to the game level(Title screen, Arrows and leaderboards)
 +
 
 +
 
 +
'''Project Objectives'''
 +
 
 +
'''1. Gestor Control:''' APDS-9960 gesture sensor on SJ2 board should respond to hand waving. I2C interface will be used for this sensor.
 +
 
 +
'''2. XBEE wireless:''' UART interface used to connect the wireless devices, which will send information to other boards.
 +
 
 +
'''3. MP3 decoder:''' MP3 data will be fetched from mini SD card reader using SPI interface.
 +
 
 +
'''4. 64x64 RGB LED matrix:''' LED will be controlled by GPIO of SJ2 board. The LED will display game graphics.
 +
 
 +
'''5. 3D Printed enclosure:''' The system will be enclosed in a 3D printed enclosure.
 +
 
 +
 
 +
 
 +
'''Team Objectives'''
 +
 
 +
'''1.'''Learn and understand the APIs of the Real Time Operating System
 +
 
 +
'''2.'''Use different embedded protocol to interface the system
 +
 
 +
'''3.'''Learn to design a module and interface the modules by reading datasheets
 +
 
 +
'''4.'''Integration and testing to deliver the final product
 +
 
 +
'''5.'''Track and respond to the reported bug and document the bugs and solutions.
 +
 
 +
== Team Members & Technical Responsibilities ==
 +
'''Jonathan Doctolero'''
 
**  Gesture Sensor Driver
 
**  Gesture Sensor Driver
 
**  Gameplay Development
 
**  Gameplay Development
 
**  Enclosure Design
 
**  Enclosure Design
*  Priyanka Rai
+
'''Priyanka Rai'''
 
**  RGB Matrix Driver
 
**  RGB Matrix Driver
**  MP3 Decoder Design
+
**  Gameplay Development
**  Wireless Driver/Application 
+
**  Wiki Report
*  Ravi Kiran Dendukuri
+
'''Ravi Kiran Dendukuri'''
**   
+
**  XBEE functionality
*  Yashwanth Kumar Nelakuditi
+
**  Gameplay Develoopment
**
+
**  PCB Design
 +
'''Yashwanth Kumar Nelakuditi'''
 +
** MP3 Decoder Design
 +
**  Gameplay Development
 +
**  Wiki Update
  
== '''Administrative Responsibilities''' ==
+
== Schedule ==
{| class="wikitable" style="margin-left: 0px; margin-right: auto;"
 
|- style="vertical-align: top;"
 
! colspan="5" style="background:#588ecc" | <span style="color:#FFFFFF"> Administrative Roles </span>
 
|-
 
! style="text-align: left;" | 
 
* <span style="color:#000000">Git Repository Manager </span>
 
|<font color="black"> Jonathan Doctolero </font>
 
|-
 
! style="text-align: left;" | 
 
* <span style="color:#000000">Wiki Report Manager</span>
 
|<font color="black"> Priyanka Rai </font>
 
|-
 
! style="text-align: left;" |
 
* <span style="color:#000000">Finance Manager </span>
 
|<font color="black"> Ravi Kiran Dendukuri </font>
 
|-
 
! style="text-align: left;" |
 
* <span style="color:#000000">Bill of Materials Manager </span>
 
|<font color="black">Yashwanth Kumar Nelakuditi </font>
 
|}
 
 
 
== '''Schedule''' ==
 
  
 
{| class="wikitable"
 
{| class="wikitable"
Line 65: Line 74:
 
! scope="col"| Task
 
! scope="col"| Task
 
! scope="col"| Status
 
! scope="col"| Status
 +
! scope="col"| Deliverable
 
|-
 
|-
 
! scope="row"| 1
 
! scope="row"| 1
 
|  
 
|  
 
* 10/12/2021
 
* 10/12/2021
* 10/13/2021
 
 
|
 
|
 
* 10/18/2021
 
* 10/18/2021
* 10/13/2021
 
 
|
 
|
* Read previous projects, gather information and discuss among the group members.
+
* Read previous projects, gather information, and discuss among the group members.
 
* Create GitLab repository for project
 
* Create GitLab repository for project
 
|  
 
|  
 
* <span style="color:green">Completed</span>
 
* <span style="color:green">Completed</span>
* <span style="color:orange">In progress</span>
+
* <span style="color:green">Completed</span>
 +
|
 +
* [https://gitlab.com/DrLer0/sjtwo-c/-/tree/FollowMeVideoGame GitLab Repo]
 
|-
 
|-
 
! scope="row"| 2
 
! scope="row"| 2
Line 86: Line 96:
 
*10/22/2021
 
*10/22/2021
 
|
 
|
 
+
*Create a Bill of Materials
 
*Order necessary parts
 
*Order necessary parts
 
|
 
|
* <span style="color:orange">In progress</span>
+
* <span style="color:green">Completed</span>
 +
* <span style="color:green">Completed</span>
 +
|
 
|-
 
|-
 
! scope="row"| 3
 
! scope="row"| 3
Line 95: Line 107:
 
*10/26/2021
 
*10/26/2021
 
|
 
|
*11/01/2021
+
*11/20/2021
 
|
 
|
 
*Read and familiarize with LED Matrix Datasheet
 
*Read and familiarize with LED Matrix Datasheet
Line 101: Line 113:
 
*Read and familiarize with XBEE Wireless UART device
 
*Read and familiarize with XBEE Wireless UART device
 
*Read and familiarize with MP3 decoder and SD card reader on SJ2
 
*Read and familiarize with MP3 decoder and SD card reader on SJ2
 +
*Design PCB
 +
|
 +
* <span style="color:green">Completed</span>
 +
* <span style="color:green">Completed</span>
 +
* <span style="color:green">Completed</span>
 +
* <span style="color:green">Completed</span>
 +
* <span style="color:green">Completed</span>
 
|
 
|
* <span style="color:red">Not started</span>
 
* <span style="color:red">Not started</span>
 
* <span style="color:red">Not started</span>
 
 
|-
 
|-
 
! scope="row"| 4
 
! scope="row"| 4
 
|
 
|
* 11/02/2021
+
*11/08/2021
 
|
 
|
* 11/08/2021
+
*11/20/2021
 +
|
 +
* Finalize wiki schedule
 +
* Order circuit boards components and complete the design for printing
 +
* Additional accessories if required and finalization of hardware
 
|
 
|
* Develop graphics driver for LED matrix and implement initial game objects
+
* <span style="color:green">Completed</span>
* Develop code module for gesture sensor
+
* <span style="color:green">Completed</span>
* Design/Plan game logic
+
* <span style="color:green">Completed</span>
 
|
 
|
* <span style="color:red">Not started</span>
+
* [http://socialledge.com/sjsu/index.php/F21:_FollowMe#PCB_Design PCB]
* <span style="color:red">Not started</span>
+
 
* <span style="color:red">Not started</span>
 
 
|-
 
|-
 
! scope="row"| 5
 
! scope="row"| 5
 
|
 
|
*11/09/2021
+
* 11/08/2021
*11/09/2021
+
|
*11/09/2021
+
* 11/20/2021
*11/09/2021
 
 
|
 
|
*11/10/2021
+
* Develop graphics driver for LED matrix and implement initial game objects
*11/15/2021
+
* Develop code module for gesture sensor
*11/15/2021
+
* Develop code module for MP3 decoder
*11/15/2021
+
* Design/Plan game logic
 
|
 
|
* Finalize wiki schedule
+
* <span style="color:green">Completed</span>
* Order circuit boards components and complete the design for printing
+
* <span style="color:green">Completed</span>
* circuit board and component assembly
+
* <span style="color:green">Completed</span>
* Circuit board testing
+
* <span style="color:green">Completed</span>
* Additional accessories if required and finalization of hardware
 
 
|
 
|
* <span style="color:red">Not started</span>
+
* [http://socialledge.com/sjsu/index.php/F21:_FollowMe#Block_Diagrams Block Diagrams]
* <span style="color:red">Not started</span>
+
* [http://socialledge.com/sjsu/index.php/F21:_FollowMe#LED_Matrix Matrix Main Screen]
* <span style="color:red">Not started</span>
 
* <span style="color:red">Not started</span>
 
* <span style="color:red">Not started</span>
 
 
|-
 
|-
 
! scope="row"| 6
 
! scope="row"| 6
Line 148: Line 162:
 
*11/16/2021
 
*11/16/2021
 
|  
 
|  
*11/22/2021
+
*11/27/2021
 
|
 
|
*Integration of circuit boards and microcontroller
+
* Receive circuit board and start component assembly
 +
* Circuit board testing
 +
* Integration of circuit boards and microcontroller
 +
* Test LED matrix API
 
* Game logic development
 
* Game logic development
 
* Testing and debugging the game logic
 
* Testing and debugging the game logic
 
|
 
|
* <span style="color:red">Not started</span>
+
* <span style="color:green">Completed</span>
* <span style="color:red">Not started</span>
+
* <span style="color:green">Completed</span>
* <span style="color:red">Not started</span>
+
* <span style="color:green">Completed</span>
 +
* <span style="color:green">Completed</span>
 +
* <span style="color:green">Completed</span>
 +
* <span style="color:green">In Progress</span>
 +
|
 +
 
 
|-
 
|-
 
! scope="row"| 7
 
! scope="row"| 7
Line 165: Line 187:
 
* 11/29/2021
 
* 11/29/2021
 
|
 
|
* Integrate game logic code with LED matrix
+
* Initial integration of game logic code with LED matrix
* Integrate game sounds with game logic
+
* Initial integration of game sounds with LED matrix
 +
* Initial integration of gesture sensor with LED matrix
 +
* Test wireless connections between 1 master and 1 slave
 +
* Debug minimum features: gesture inputs, game display, music playing, single player
 
|
 
|
* <span style="color:red">Not started</span>
+
* <span style="color:green">Completed</span>
* <span style="color:red">Not started</span>
+
* <span style="color:green">Completed</span>
 +
* <span style="color:green">Completed</span>
 +
* <span style="color:green">Completed</span>
 +
* <span style="color:green">Completed</span>
 +
|
 +
 
 
|-
 
|-
 
! scope="row"| 8
 
! scope="row"| 8
Line 178: Line 208:
 
|
 
|
  
* Integrate subsystem
+
* Test full integration of LED matrix, MP3 decoder, gesture sensor, and wireless communication
* Finalizing the video game
+
* Add multiple players, ensure each device can connect to the network
 
* Update the wiki page.
 
* Update the wiki page.
 
|
 
|
* <span style="color:red">Not started</span>
+
* <span style="color:green">Completed</span>
* <span style="color:red">Not started</span>
+
* <span style="color:green">Completed</span>
* <span style="color:red">Not started</span>
+
* <span style="color:green">Completed</span>
 +
|
 +
[http://socialledge.com/sjsu/index.php/F21:_FollowMe#Implementation Integration to be placed in 3D printed case]
 
|-
 
|-
 
! scope="row"| 9
 
! scope="row"| 9
Line 193: Line 225:
 
|
 
|
 
* Address bugs during testing of the integrated system
 
* Address bugs during testing of the integrated system
* Test pause/play functionality
+
* Thorough testing of the game
 +
* Debug multiple players
 +
|
 +
* <span style="color:green">Completed</span>
 +
* <span style="color:green">Completed</span>
 +
* <span style="color:green">Completed</span>
 
|
 
|
* <span style="color:red">Not started</span>
+
 
* <span style="color:red">Not started</span>
 
* <span style="color:red">Not started</span>
 
 
|-
 
|-
 
! scope="row"| 10
 
! scope="row"| 10
 
|  
 
|  
*12/16/2021
+
*12/15/2021
*12/14/2021
 
*12/14/2021
 
*12/14/2021
 
 
|  
 
|  
*12/16/2021
+
 
*12/16/2021
 
*12/16/2021
 
*12/16/2021
 
 
|
 
|
  
Line 217: Line 246:
 
*Update the wiki page.
 
*Update the wiki page.
 
|
 
|
* <span style="color:red">Not started</span>
+
* <span style="color:green">Completed</span>
* <span style="color:red">Not started</span>
+
* <span style="color:green">Completed</span>
* <span style="color:red">Not started</span>
+
* <span style="color:green">Completed</span>
* <span style="color:red">Not started</span>
+
* <span style="color:green">Completed</span>
 +
|
 +
 
 
|-
 
|-
 +
! scope="row"| 11
 +
|
 +
*12/16/2021
 +
|
 +
 +
|
 +
 +
* Turn in Wiki Page Report
 +
|
 +
* <span style="color:green">Completed</span>
 
|}
 
|}
  
<HR>
 
 
<BR/>
 
<BR/>
  
== Parts List & Cost ==
+
== Bill of Materials (General Parts) ==
Give a simple list of the cost of your project broken down by components. Do not write long stories here.
+
 
 +
{| class="wikitable"
 +
|-
 +
! scope="col" style="text-align: left;" style="background:#588ecc;" | <span style="color:#FFFFFF"> PART NAME </span>
 +
 
 +
! scope="col" style="text-align: left;" style="background:#588ecc;" | <span style="color:#FFFFFF">
 +
PART MODEL & SOURCE </span>
 +
! scope="col" style="text-align: left;" style="background:#588ecc;" | <span style="color:#FFFFFF">
 +
QUANTITY </span>
 +
! scope="col" style="text-align: left;" style="background:#588ecc;" | <span style="color:#FFFFFF">
 +
COST PER UNIT (USD) </span>
 +
|-
 +
|
 +
* Micro-Controller Eval-Boards
 +
|
 +
* LPC 4078 (Purchased from Preet Kang)
 +
|
 +
* 3
 +
|
 +
* 50.00
 +
 
 +
|-
 +
|
 +
* RGB LED matrix
 +
|
 +
* [https://www.sparkfun.com/products/14718 32 x 64 RGB LED Matrix Panel]
 +
|
 +
* 1
 +
|
 +
* 49.95
 +
 
 +
|-
 +
|
 +
* Power supply
 +
|
 +
* [https://www.sparkfun.com/products/15352 5V / 4A Power Supply]
 +
|
 +
* 1
 +
|
 +
* 12.95
 +
 
 +
|-
 +
|
 +
* Xbee Adapter
 +
|
 +
* Give the link
 +
|
 +
* 1
 +
|
 +
* 5
 +
 
 +
|-
 +
|
 +
* Xbee Transreceiver
 +
|
 +
* Give the link
 +
|
 +
* 3
 +
|
 +
* Borrowed from Preet
 +
 
 +
|-
 +
|
 +
* MP3 music player (YX5300)
 +
|
 +
* Amazon
 +
|
 +
* 1
 +
|
 +
* 8
 +
 
 +
|-
 +
|
 +
* PCB Fabrication
 +
|
 +
* [https://jlcpcb.com/quote JLCPCB]
 +
|
 +
* 5
 +
|
 +
* 29.53
 +
 
 +
|-
 +
|
 +
* Miscellaneous parts
 +
|
 +
* Anchor Electronics and Digikey '''
 +
|
 +
* 1
 +
|
 +
* 50.00
 +
|}
 +
 
 +
<BR/>
  
 
== Design & Implementation ==
 
== Design & Implementation ==
The design section can go over your hardware and software design.  Organize this section using sub-sections that go over your design and implementation.
 
  
=== Hardware Design ===
+
=== PCB Design ===
Discuss your hardware design here. Show detailed schematics and the interface here.
+
We chose EasyEDA for PCB design,it is an online free software.
 +
 
 +
==== Schematic Design: ====
 +
 
 +
Our project is divided into two main circuits, first is main and master controller circuit and second is a slave controller circuit.  
 +
 
 +
 
 +
[[File:master.png|800px|thumb|center|'''Master Controller PCB Schematic''']]
 +
 
 +
[[File:slave1.png|700px|thumb|center|'''Slave Controller PCB Schematic''']]
 +
 
  
=== Hardware Interface ===
+
*Mater circuit will do the wire interfacing of SJTwo board to xbee, mp3 decoder, led matrix. This is the master board and it will recieve values from all the controllers and peripherals
In this section, you can describe how your hardware communicates, such as which BUSes used.  You can discuss your driver implementation here, such that the '''Software Design''' section is isolated to talk about high level workings rather than inner working of your project.
+
 
 +
*Slave circuit will do the wire interfacing of SJTwo board to xbee  which will send the values of Gesture to the master Xbee.
 +
 
 +
==== PCB Layout: ====
 +
 
 +
With EasyEDa we can conver schematic to PCB it auto routes itself.
 +
 
 +
<center>
 +
<table>
 +
<tr>
 +
<td>
 +
[[File:PCB.png|300px|thumb|center|'''PCB Schematic 3D view''']]
 +
</td>
 +
<td>
 +
[[File:PCb3.jpg|300px|thumb|center|'''Printed PCB''']]
 +
</td>
 +
<td>
 +
</tr>
 +
</table>
 +
</center>
 +
 
 +
=== Hardware Interface===
 +
 
 +
<center>
 +
<table>
 +
<tr>
 +
<td>
 +
[[File:flow.png|550px|thumb|center|'''Main Controller''']]
 +
</td>
 +
<td>
 +
[[File:flow2.png|250px|thumb|center|'''Slave Controller''']]
 +
</td>
 +
<td>
 +
</tr>
 +
</table>
 +
</center>
 +
 
 +
Hardware Interface gives an overview of the entire system which consists of the two SJ2 controllers, one board is used as Main/Master controller and other as Slave controller.
 +
 
 +
*The Slave controller uses the onboard Gesture on SJ2 which is interfaced via UART protocol.
 +
 
 +
*The Main controller board is used to control LED Matrix.This matrix displays the Arrows and accordingly player should play and value received from the slave through Wireless Module is used to determine whether it is right or wrong.
 +
 
 +
=== Hardware Components Implementation ===
 +
 
 +
==== LED Matrix ====
 +
 
 +
 
 +
A 64 x 64 RGB LED Matrix will be powered up through a 5V/4A DC adapter and is interfaced with the board to play the game '''FOLLOW ME''' with the background music. The matrix has 2 planes (upper and lower), both of which will be programmed separately. Planes are made by dividing 64 rows into 2 halves, i.e. first 32 rows in plane 1 and the remaining 32 rows in plane 2. Five signals (viz. A, B, C, D, and E) which are connected to 5 GPIOs of the SJ-2 board are used to select rows from each plane. R1, G1, B1 signals with 64-bit registers are used to address individual led from the selected row in plane 1. Similarly, R2, G2, B2 signals used to address individual led from the selected row in plane 2.
 +
A single clock is interfaced to these 6 64bit shift registers. Hence, at one clock signal, we fill and enable a column corresponding to two selected rows. Once the clocking and register shifting is done, the data need to be latched to the register which in turn enables the corresponding LED. Then the LATCH is set to mark end of the row and reset to move to next row. The output enable signal is enabled to push the data to the selected row. The output enable signal is disabled again to select the other row. All of these steps are repeated at very less time intervals so that the human eye perceives it as one complete frame (Persistence of Vision).
 +
<center>
 +
<table>
 +
<tr>
 +
<td>
 +
[[File:2048_ledmatrix_2.jpg|600px|thumb|left|LED Matrix Schematic]]
 +
</td>
 +
<td>
 +
{| class="wikitable"
 +
|-
 +
! scope="col"| PIN
 +
! scope="col"| Pin Desciption
 +
! scope="col"| SJ2 PIN
 +
|-
 +
! scope="row"| R1
 +
| PIN for Red terminal of RGB LED for upper half of LED Matrix
 +
| P2_0
 +
|-
 +
! scope="row"| G1
 +
| PIN for Green terminal of RGB LED for upper half of LED Matrix
 +
| P2_1
 +
|-
 +
! scope="row"| B1
 +
| PIN for Blue terminal of RGB LED for upper half of LED Matrix
 +
| P2_2
 +
|-
 +
! scope="row"| R2
 +
| PIN for Red terminal of RGB LED for lower half of LED Matrix
 +
| P2_4
 +
|-
 +
! scope="row"| G2
 +
| PIN for Green terminal of RGB LED for lower half of LED Matrix
 +
| P2_5
 +
|-
 +
! scope="row"| B2
 +
| PIN for Blue terminal of RGB LED for lower half of LED Matrix
 +
| P2_6
 +
|-
 +
! scope="row"| A
 +
|  Mux pin for row selection
 +
| P2_7
 +
|-
 +
! scope="row"| B
 +
|  Mux pin for row selection
 +
| P2_8
 +
|-
 +
! scope="row"| C
 +
|  Mux pin for row selection
 +
| P2_9
 +
|-
 +
! scope="row"| D
 +
|  Mux pin for row selection
 +
| P0_16
 +
|-
 +
! scope="row"| E
 +
|  Mux pin for row selection
 +
| P0_15
 +
|-
 +
! scope="row"| OE
 +
| Output Enable
 +
| P1_20
 +
|-
 +
! scope="row"| LATCH
 +
|  Latching Data
 +
| P1_23
 +
|-
 +
! scope="row"| CLK
 +
|  CLock signal to understand one bit has been detected
 +
| P1_28
 +
|-
 +
 
 +
! scope="row"| GND
 +
| Ground
 +
| GND
 +
|-
 +
|}
 +
</td>
 +
<td>
 +
</tr>
 +
</table>
 +
</center>
 +
 
 +
==== Gesture ====
 +
[[File:GestureEngine.png | 400px]]
 +
 
 +
[https://gitlab.com/DrLer0/sjtwo-c/-/merge_requests/11 Gesutre Implementation]
 +
 
 +
==== Gesture Sensor Code Module based on APDS-9960 ====
 +
Gesture Sensor is initialized at startup in entry_point() via `sensors_init()`. To obtain gesture data use `#include "apds.h"` and use `apds__get_gesture()` to return a `apds_gesture_s` type.
 +
 
 +
# Example
 +
 
 +
  void get_gesture_task(void *params) {
 +
    apds__gesture_s userInput = ERROR;
 +
    while (1) {
 +
      userInput = apds__get_gesture();
 +
      if (userInput != ERROR) {
 +
        printf("userInput = %c\n", userInput);
 +
      }
 +
      vTaskDelay(1000);
 +
    }
 +
  }
 +
 
 +
A task polls to check if a gesture event occured. If so, it'll print 'U', 'D', 'L', 'R', 'N', or 'F' according to the gesture that was detected and then sleep for 1 second. If no gesture event occured, then the task will sleep for 1 second.
 +
 
 +
# Example Use Case:
 +
 
 +
  If there was a valid gesture detected, then send the user input to a queue. Another task will read from that queue and change states according to the user input, if there is no data on the queue, then don't change states.
 +
 
 +
==== Detailed Explanation how the Gesture Sensor Code Module works ====
 +
 
 +
1. APDS-9960 will update its Gesture Status Register (GSTATUS: 0xAF) if there is valid data in the gesture FIFOs for Up, Down, Left, and Right IR photodiodes on the sensor.
 +
 
 +
2. Bit 0 of GSTATUS will be 1 if there is valid data, else 0 if there is no gesture data collected.
 +
 
 +
3. If GSTATUS[0] == 1, then read the FIFO level from the Gesture FIFO Level Register (GFLVL: 0xAE). Values in GFLVL will be between 0 and 32, inclusively. The value indicates how many sets of bytes are in the FIFO to be read. For example, if GFLVl == 32, then you have to read 32*4 bytes from the FIFO registers.
 +
 
 +
4. Start I2C read of GFLVL*4 bytes starting at Gesture FIFO Up Register (GFIFO_U: 0xFC). Save the data to an array to be processed. Data is represented in values 0 to 255, as seen in Fig 1 below.
 +
 
 +
5. Gesture data is set up in pairs: Up-Down and Left-Right. Therefore, check which pair has the largest difference.
 +
 
 +
6. Then check which photodiode of the pair has the smallest value. That photodiode was passed over first. Record that photodiode in an array.
 +
 
 +
7. Once all the raw gesture data is processed and you have a series of photodiodes saved in an array. Process that photodiode array. Photodiode array is represented in the second to last line of Fig 1 below.
 +
 
 +
8. At the first index, check if the next photodiode is part of the pair. For example: photodiode_array[0] == U and photodiode_array[1] == D, then the gesture that occured was a swipe down. Return 'D' for down.
 +
 
 +
==== XBee ====
 +
 
 +
[[File:xbee_module.jpg|350px|thumb|center|'''XBee S1''']]
 +
For wireless communication we utilized the XBee S1 module. This module provides a "transparent" UART interface to the Master and Controller boards. The XBee S1 is designed by Digi Inc. and allows for relatively easy programming via the XTCU programming suite using a XBee USB programmer. This device internally uses 802.15.4 to wirelessly send UART frames between modules. To communicate with the master and slave, the project employs two Xbees. The Master gives the Slave the direction and reverse mode, which is used to check if the input gesture is in the same direction as the master. The master receives confirmation of this, which is used to increase the player's score.
 +
 
 +
===== XCTU configuration =====
 +
 
 +
<center>
 +
<table>
 +
<tr>
 +
<td>
 +
[[File:xbee.png|700px|thumb|center|'''pin configuration from sj2 to XBee''']]
 +
</td>
 +
<td>
 +
{| class="wikitable"
 +
|-
 +
! scope="col"| XCTU
 +
! scope="col"| Transmitter
 +
! scope="col"| Receiver
 +
|-
 +
! scope="row"| CH
 +
| C
 +
| C
 +
|-
 +
! scope="row"| ID
 +
| 1001
 +
| 1001
 +
|-
 +
! scope="row"| CE
 +
| Coordinator
 +
| Endpoint
 +
|-
 +
! scope="row"| Baud Rate
 +
| 9600
 +
| 9600
 +
|-
 +
|}
 +
</td>
 +
<td>
 +
</tr>
 +
</table>
 +
</center>
 +
 
 +
''' Sample Code snippet for Sender task '''
 +
 
 +
<syntaxhighlight lang="c">
 +
void board_1_sender_task(void *p) {
 +
  char userInput = NULL;
 +
  char current_arrow = NULL;
 +
  char send_correct = 'C';
 +
  while (true) {
 +
    if (xQueueReceive(player_input_queue, &userInput, 800)) {
 +
      if (xQueueReceive(master_input_task_queue, &current_arrow, 1000)) {
 +
        if (current_arrow == userInput) {
 +
          uart__put(UART__3, send_correct, 0);
 +
        }
 +
      }
 +
    }
 +
  }
 +
}
 +
 
 +
 
 +
</syntaxhighlight>
 +
 
 +
==== MP3 Decoder====
 +
 
 +
The module is a kind of simple MP3 player device which is based on a high-quality MP3 audio chip-YX5300. It can support 8k Hz ~ 48k Hz sampling frequency MP3 and WAV file formats.
 +
There is a TF card socket on board, so you can plug the micro SD card that stores audio files. MCU can control the MP3 playback state by sending commands to the module via UART port, such as switch songs, change the volume and play mode and so on. You can also debug the module via USB to UART module. The SD card should be formatted as FAT16 or FAT32 and should have some audio files with .mp3 or .wav formats. If user wants to create separate folders then those should be created as “01”, “02” and the songs should be with the names 001xxx.mp3/ 002xxx.mp3/ 003xxx.mp3 in those created folders.
 +
<center>
 +
<table>
 +
<tr>
 +
<td>
 +
[[File:MP3_Decoder_board.JPG|300px|thumb|center| MP3 Decoder]]
 +
</td>
 +
<td>
 +
[[File:music1.png|700px|thumb|center|pin configuration of SJ2 board to MP3.]]
 +
</td>
 +
</tr>
 +
</table>
 +
</center>
 +
 
 +
''' Code snippet for MP3 '''
 +
 
 +
<syntaxhighlight lang="c">
 +
 
 +
bool mp3__send_command(uint8_t command, uint16_t data) {
 +
  bool status = false;
 +
  uint8_t data_ub = (uint8_t)(data >> 8);
 +
  uint8_t data_lb = (uint8_t)(data);
 +
  mp3_uart_buffer[0] = 0x7e;
 +
  mp3_uart_buffer[1] = 0xff;
 +
  mp3_uart_buffer[2] = 0x06;
 +
  mp3_uart_buffer[3] = command;
 +
  mp3_uart_buffer[4] = 0x00;
 +
  mp3_uart_buffer[5] = data_ub;
 +
  mp3_uart_buffer[6] = data_lb;
 +
  mp3_uart_buffer[7] = 0xef;
 +
 
 +
  for (int i = 0; i < 8; i++) {
 +
    uart__polled_put(UART__3, mp3_uart_buffer[i]);
 +
  }
 +
  return status = true;
 +
}
 +
</syntaxhighlight>
 +
 
 +
[[File:music.png|400px|thumb|center|MP3 Flow]]
  
 
=== Software Design ===
 
=== Software Design ===
Show your software design. For example, if you are designing an MP3 Player, show the tasks that you are using, and what they are doing at a high level. Do not show the details of the code. For example, do not show exact code, but you may show psuedocode and fragments of code. Keep in mind that you are showing DESIGN of your software, not the inner workings of it.
+
 
 +
1. <font color="black"> '''Timer - Mode''' </font>
 +
 
 +
a) Set the timer to finish the game in given time
 +
 
 +
b) Arrows change every five seconds
 +
 
 +
 
 +
<font color="black"> '''Game rules:''' </font>
 +
 
 +
a) Game comprises of 3 levels (level 2 on reaching score 15 and level 3 on reaching score 30).
 +
 
 +
b) Primary objective is to move hands in direction of arrow if the color is GREEN and in opposite direction if the color is RED.
 +
 
 +
c) Score increases by 1 point for right hand movement.
 +
 
 +
d) Level 3 is faster so make sure you tune your reflexes up.
 +
 
 +
e) Reach score 50 to win the game.
 +
 
 +
 
 +
There are 2 tasks involved to ensure functioning of the game.
 +
 
 +
a) Task 1:
 +
*Display the game's title screen. The game begins with receiving gesture from any of the player.
 +
*Also used to display game over screen / win screen.
 +
b) Task 2:
 +
*Enter into game mode to play the game.
 +
*Generate arrow matrix frame for random direction(horizontal, vertical, up and down) and random color(Red/Green).
 +
*Detect the player's gesture from his board, compare it with arrow's location and color.
 +
*Display score and level.
 +
*Upon receiving Far gesture from the player, stop game and return to title screen or to restart the game once it is over.
  
 
=== Implementation ===
 
=== Implementation ===
This section includes implementation, but again, not the details, just the high level. For example, you can list the steps it takes to communicate over a sensor, or the steps needed to write a page of memory onto SPI Flash. You can include sub-sections for each of your component implementations.
+
[https://gitlab.com/DrLer0/sjtwo-c/-/merge_requests/13 Follow Me Game Files]
 +
 
 +
[[File:GameCasewithMatrix.png | 400px]] [[File:GameCase.png | 400px]]
 +
 
 +
In the figures above, one SJ2 board will be connected to the PCB to connect to the MP3 decoder, XBEE, and LED matrix and placed inside the 3D printed game case.
 +
 
 +
[[File:PlayerCasewithLid.png | 400px]] [[File:PlayerCase.png | 400px]]
 +
 
 +
In the figures above, another SJ2 board connected to an XBEE will be placed in a player case with the gesture sensor exposed for players to swipe their hands above it.
 +
 
 +
 
 +
[[File:PackagedUp.jpeg | 500px]]
  
 
== Testing & Technical Challenges ==
 
== Testing & Technical Challenges ==
Describe the challenges of your project.  What advice would you give yourself or someone else if your project can be started from scratch again?
 
Make a smooth transition to the testing section and describe what it took to test your project.
 
  
Include sub-sections that list out a problem and solution, such as:
+
=== RGB LED Matrix ===
 +
 
 +
1. <font color="black"> '''RGB LED Matrix'''  </font>
 +
 
 +
* The ordered matrix was faulty, needed to order another one, project start got delayed a bit due to this.
 +
 
 +
* Getting control of the matrix was challenging as it needs to follow specific sequence of pin enabling/disabling at appropriate timing.
 +
 
 +
* Setting right delays to avoid flickers and get smooth transitioning between frames by using delays or for loops.
  
=== <Bug/issue name> ===
+
* Designing elements of the game (title screen and arrow objects) which required careful plotting of pixel data.
Discuss the issue and resolution.
 
  
 
== Conclusion ==
 
== Conclusion ==
Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?
+
It was a really great working experience for all of us. We worked as a team of four which helped us all to know the importance of team-work and co-ordination. We learned many great stuff like
 +
 
 +
* State design using FreeRTOS APIs.
 +
* Uses of static, extern and global variables.
 +
* Learning optimization through the integration of individual modules
 +
* Task priorities can effect working of overall project.
 +
* Workload sharing and cooperation amongst peers.
 +
* Sharing ideas and debugging a common problem together i.e team building.
 +
* Hardware and it's design plays important role in development and presentation of the project.
  
 +
There were a lot of problems while doing this project but it was a steep learning curve. In the end we were able to achieve our objective with really good results.
 
=== Project Video ===
 
=== Project Video ===
Upload a video of your project and post the link here.
+
https://youtu.be/N68ExnJIXpk
  
 
=== Project Source Code ===
 
=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]
+
* Git Project Link: <font color="blue"><U>[https://gitlab.com/DrLer0/sjtwo-c/-/tree/FollowMeVideoGame]</U></font>
 +
* Git Project Link(individual modules): <font color="blue"><U>[https://gitlab.com/DrLer0/sjtwo-c/-/tree/FollowMeVideoGame]</U></font>
  
== References ==
 
 
=== Acknowledgement ===
 
=== Acknowledgement ===
Any acknowledgment that you may wish to provide can be included here.
+
We would like to express our gratitude to Professor Preetpal Kang for generously sharing his time and knowledge with us and guiding us through the completion of this project. We would also like to thank our ISA Tirth Pandya for his valuable advice and constructive feedback.
 +
 
 +
== References Used ==
 +
 
 +
====== RGB LED Matrix Interfacing and Designing ======
 +
*  [https://www.sparkfun.com/products/14718 Datasheet and Hookup guide]
 +
*  [https://cdn-learn.adafruit.com/downloads/pdf/32x16-32x32-rgb-led-matrix.pdf?timestamp=1543806512/ Wiring and pin information]
 +
*  Hex code generator to build the led matrix [https://www.prf.jcu.cz/generator-led-matrix/index.htm]
 +
 
 +
====== MP3 Decoder ======
 +
*  [http://geekmatic.in.ua/pdf/Catalex_MP3_board.pdf Datasheet For MP3 Decoder]
  
=== References Used ===
+
====== Gesture & Orientation Sensors  ======
List any references used in project.
+
*  [https://cdn.sparkfun.com/assets/learn_tutorials/3/2/1/Avago-APDS-9960-datasheet.pdf Gesture Sensor Data Sheet]
 +
*  [https://learn.sparkfun.com/tutorials/mma8452q-accelerometer-breakout-hookup-guide/all Accelerometer Sensor Data]
  
=== Appendix ===
+
====== General/Miscellaneous ======
You can list the references you used.
+
*  [http://socialledge.com/sjsu/index.php/FreeRTOS_Tutorial FreeRTOS Tutorial]
 +
*  [https://sjsu-dev2.readthedocs.io/en/latest/?badge=latest SJTwo-c Documentation]

Latest revision as of 01:56, 18 December 2021

Abstract

FollowMe is a video game where you follow the correct directions. The LED matrix will display a series of arrows and the player needs to correctly wave their hand over their gesture sensor to earn points. However, watch out for the background color of the arrow. If the background is green, then the players need to wave their hand in that direction; i.e, an arrow pointing left with a green background means wave your hand from the right to the left (wave left). If the background is red, then the players need to wave their hand in the opposite direction; i.e, an arrow pointing up with a red background means wave your hand from the top to the bottom (wave down).

The game has a timer mode. The player tries to get the most correct waves until the time runs out and the arrows change every five seconds.

Block Diagrams

MainScreen.png GameMode.png

PlayerInput.png PlayerGame.png

Objectives & Introduction

Project Introduction

1.Analyse hand waving/movement using a gesture sensor

2.Connect different microcontrollers wirelessly to exchange information

3.Read a file from SD card on the microcontroller and get the MP3 data using MP3 decoder

4.Display different screens according to the game level(Title screen, Arrows and leaderboards)


Project Objectives

1. Gestor Control: APDS-9960 gesture sensor on SJ2 board should respond to hand waving. I2C interface will be used for this sensor.

2. XBEE wireless: UART interface used to connect the wireless devices, which will send information to other boards.

3. MP3 decoder: MP3 data will be fetched from mini SD card reader using SPI interface.

4. 64x64 RGB LED matrix: LED will be controlled by GPIO of SJ2 board. The LED will display game graphics.

5. 3D Printed enclosure: The system will be enclosed in a 3D printed enclosure.


Team Objectives

1.Learn and understand the APIs of the Real Time Operating System

2.Use different embedded protocol to interface the system

3.Learn to design a module and interface the modules by reading datasheets

4.Integration and testing to deliver the final product

5.Track and respond to the reported bug and document the bugs and solutions.

Team Members & Technical Responsibilities

  • Jonathan Doctolero
    • Gesture Sensor Driver
    • Gameplay Development
    • Enclosure Design
  • Priyanka Rai
    • RGB Matrix Driver
    • Gameplay Development
    • Wiki Report
  • Ravi Kiran Dendukuri
    • XBEE functionality
    • Gameplay Develoopment
    • PCB Design
  • Yashwanth Kumar Nelakuditi
    • MP3 Decoder Design
    • Gameplay Development
    • Wiki Update

Schedule

Week# Start Date End Date Task Status Deliverable
1
  • 10/12/2021
  • 10/18/2021
  • Read previous projects, gather information, and discuss among the group members.
  • Create GitLab repository for project
  • Completed
  • Completed
2
  • 10/19/2021
  • 10/22/2021
  • Create a Bill of Materials
  • Order necessary parts
  • Completed
  • Completed
3
  • 10/26/2021
  • 11/20/2021
  • Read and familiarize with LED Matrix Datasheet
  • Read and familiarize with APDS-9960 datasheet
  • Read and familiarize with XBEE Wireless UART device
  • Read and familiarize with MP3 decoder and SD card reader on SJ2
  • Design PCB
  • Completed
  • Completed
  • Completed
  • Completed
  • Completed
4
  • 11/08/2021
  • 11/20/2021
  • Finalize wiki schedule
  • Order circuit boards components and complete the design for printing
  • Additional accessories if required and finalization of hardware
  • Completed
  • Completed
  • Completed
5
  • 11/08/2021
  • 11/20/2021
  • Develop graphics driver for LED matrix and implement initial game objects
  • Develop code module for gesture sensor
  • Develop code module for MP3 decoder
  • Design/Plan game logic
  • Completed
  • Completed
  • Completed
  • Completed
6
  • 11/16/2021
  • 11/27/2021
  • Receive circuit board and start component assembly
  • Circuit board testing
  • Integration of circuit boards and microcontroller
  • Test LED matrix API
  • Game logic development
  • Testing and debugging the game logic
  • Completed
  • Completed
  • Completed
  • Completed
  • Completed
  • In Progress
7
  • 11/23/2021
  • 11/29/2021
  • Initial integration of game logic code with LED matrix
  • Initial integration of game sounds with LED matrix
  • Initial integration of gesture sensor with LED matrix
  • Test wireless connections between 1 master and 1 slave
  • Debug minimum features: gesture inputs, game display, music playing, single player
  • Completed
  • Completed
  • Completed
  • Completed
  • Completed
8
  • 11/30/2021
  • 12/06/2021
  • Test full integration of LED matrix, MP3 decoder, gesture sensor, and wireless communication
  • Add multiple players, ensure each device can connect to the network
  • Update the wiki page.
  • Completed
  • Completed
  • Completed

Integration to be placed in 3D printed case

9
  • 12/07/2021
  • 12/13/2021
  • Address bugs during testing of the integrated system
  • Thorough testing of the game
  • Debug multiple players
  • Completed
  • Completed
  • Completed
10
  • 12/15/2021
  • Final Demo
  • Update Gitlab repo with final code.
  • Update test video.
  • Update the wiki page.
  • Completed
  • Completed
  • Completed
  • Completed
11
  • 12/16/2021
  • Turn in Wiki Page Report
  • Completed


Bill of Materials (General Parts)

PART NAME

PART MODEL & SOURCE

QUANTITY

COST PER UNIT (USD)

  • Micro-Controller Eval-Boards
  • LPC 4078 (Purchased from Preet Kang)
  • 3
  • 50.00
  • RGB LED matrix
  • 1
  • 49.95
  • Power supply
  • 1
  • 12.95
  • Xbee Adapter
  • Give the link
  • 1
  • 5
  • Xbee Transreceiver
  • Give the link
  • 3
  • Borrowed from Preet
  • MP3 music player (YX5300)
  • Amazon
  • 1
  • 8
  • PCB Fabrication
  • 5
  • 29.53
  • Miscellaneous parts
  • Anchor Electronics and Digikey
  • 1
  • 50.00


Design & Implementation

PCB Design

We chose EasyEDA for PCB design,it is an online free software.

Schematic Design:

Our project is divided into two main circuits, first is main and master controller circuit and second is a slave controller circuit.


Master Controller PCB Schematic
Slave Controller PCB Schematic


  • Mater circuit will do the wire interfacing of SJTwo board to xbee, mp3 decoder, led matrix. This is the master board and it will recieve values from all the controllers and peripherals
  • Slave circuit will do the wire interfacing of SJTwo board to xbee which will send the values of Gesture to the master Xbee.

PCB Layout:

With EasyEDa we can conver schematic to PCB it auto routes itself.

PCB Schematic 3D view
Printed PCB

Hardware Interface

Main Controller
Slave Controller

Hardware Interface gives an overview of the entire system which consists of the two SJ2 controllers, one board is used as Main/Master controller and other as Slave controller.

  • The Slave controller uses the onboard Gesture on SJ2 which is interfaced via UART protocol.
  • The Main controller board is used to control LED Matrix.This matrix displays the Arrows and accordingly player should play and value received from the slave through Wireless Module is used to determine whether it is right or wrong.

Hardware Components Implementation

LED Matrix

A 64 x 64 RGB LED Matrix will be powered up through a 5V/4A DC adapter and is interfaced with the board to play the game FOLLOW ME with the background music. The matrix has 2 planes (upper and lower), both of which will be programmed separately. Planes are made by dividing 64 rows into 2 halves, i.e. first 32 rows in plane 1 and the remaining 32 rows in plane 2. Five signals (viz. A, B, C, D, and E) which are connected to 5 GPIOs of the SJ-2 board are used to select rows from each plane. R1, G1, B1 signals with 64-bit registers are used to address individual led from the selected row in plane 1. Similarly, R2, G2, B2 signals used to address individual led from the selected row in plane 2. A single clock is interfaced to these 6 64bit shift registers. Hence, at one clock signal, we fill and enable a column corresponding to two selected rows. Once the clocking and register shifting is done, the data need to be latched to the register which in turn enables the corresponding LED. Then the LATCH is set to mark end of the row and reset to move to next row. The output enable signal is enabled to push the data to the selected row. The output enable signal is disabled again to select the other row. All of these steps are repeated at very less time intervals so that the human eye perceives it as one complete frame (Persistence of Vision).

LED Matrix Schematic
PIN Pin Desciption SJ2 PIN
R1 PIN for Red terminal of RGB LED for upper half of LED Matrix P2_0
G1 PIN for Green terminal of RGB LED for upper half of LED Matrix P2_1
B1 PIN for Blue terminal of RGB LED for upper half of LED Matrix P2_2
R2 PIN for Red terminal of RGB LED for lower half of LED Matrix P2_4
G2 PIN for Green terminal of RGB LED for lower half of LED Matrix P2_5
B2 PIN for Blue terminal of RGB LED for lower half of LED Matrix P2_6
A Mux pin for row selection P2_7
B Mux pin for row selection P2_8
C Mux pin for row selection P2_9
D Mux pin for row selection P0_16
E Mux pin for row selection P0_15
OE Output Enable P1_20
LATCH Latching Data P1_23
CLK CLock signal to understand one bit has been detected P1_28
GND Ground GND

Gesture

GestureEngine.png

Gesutre Implementation

Gesture Sensor Code Module based on APDS-9960

Gesture Sensor is initialized at startup in entry_point() via `sensors_init()`. To obtain gesture data use `#include "apds.h"` and use `apds__get_gesture()` to return a `apds_gesture_s` type.

  1. Example
 void get_gesture_task(void *params) {
   apds__gesture_s userInput = ERROR;
   while (1) {
     userInput = apds__get_gesture();
     if (userInput != ERROR) {
       printf("userInput = %c\n", userInput);
     }
     vTaskDelay(1000);
   }
 }

A task polls to check if a gesture event occured. If so, it'll print 'U', 'D', 'L', 'R', 'N', or 'F' according to the gesture that was detected and then sleep for 1 second. If no gesture event occured, then the task will sleep for 1 second.

  1. Example Use Case:
 If there was a valid gesture detected, then send the user input to a queue. Another task will read from that queue and change states according to the user input, if there is no data on the queue, then don't change states.

Detailed Explanation how the Gesture Sensor Code Module works

1. APDS-9960 will update its Gesture Status Register (GSTATUS: 0xAF) if there is valid data in the gesture FIFOs for Up, Down, Left, and Right IR photodiodes on the sensor.

2. Bit 0 of GSTATUS will be 1 if there is valid data, else 0 if there is no gesture data collected.

3. If GSTATUS[0] == 1, then read the FIFO level from the Gesture FIFO Level Register (GFLVL: 0xAE). Values in GFLVL will be between 0 and 32, inclusively. The value indicates how many sets of bytes are in the FIFO to be read. For example, if GFLVl == 32, then you have to read 32*4 bytes from the FIFO registers.

4. Start I2C read of GFLVL*4 bytes starting at Gesture FIFO Up Register (GFIFO_U: 0xFC). Save the data to an array to be processed. Data is represented in values 0 to 255, as seen in Fig 1 below.

5. Gesture data is set up in pairs: Up-Down and Left-Right. Therefore, check which pair has the largest difference.

6. Then check which photodiode of the pair has the smallest value. That photodiode was passed over first. Record that photodiode in an array.

7. Once all the raw gesture data is processed and you have a series of photodiodes saved in an array. Process that photodiode array. Photodiode array is represented in the second to last line of Fig 1 below.

8. At the first index, check if the next photodiode is part of the pair. For example: photodiode_array[0] == U and photodiode_array[1] == D, then the gesture that occured was a swipe down. Return 'D' for down.

XBee

XBee S1

For wireless communication we utilized the XBee S1 module. This module provides a "transparent" UART interface to the Master and Controller boards. The XBee S1 is designed by Digi Inc. and allows for relatively easy programming via the XTCU programming suite using a XBee USB programmer. This device internally uses 802.15.4 to wirelessly send UART frames between modules. To communicate with the master and slave, the project employs two Xbees. The Master gives the Slave the direction and reverse mode, which is used to check if the input gesture is in the same direction as the master. The master receives confirmation of this, which is used to increase the player's score.

XCTU configuration
pin configuration from sj2 to XBee
XCTU Transmitter Receiver
CH C C
ID 1001 1001
CE Coordinator Endpoint
Baud Rate 9600 9600

Sample Code snippet for Sender task

void board_1_sender_task(void *p) {
  char userInput = NULL;
  char current_arrow = NULL;
  char send_correct = 'C';
  while (true) {
    if (xQueueReceive(player_input_queue, &userInput, 800)) {
      if (xQueueReceive(master_input_task_queue, &current_arrow, 1000)) {
        if (current_arrow == userInput) {
          uart__put(UART__3, send_correct, 0);
        }
      }
    }
  }
}

MP3 Decoder

The module is a kind of simple MP3 player device which is based on a high-quality MP3 audio chip-YX5300. It can support 8k Hz ~ 48k Hz sampling frequency MP3 and WAV file formats. There is a TF card socket on board, so you can plug the micro SD card that stores audio files. MCU can control the MP3 playback state by sending commands to the module via UART port, such as switch songs, change the volume and play mode and so on. You can also debug the module via USB to UART module. The SD card should be formatted as FAT16 or FAT32 and should have some audio files with .mp3 or .wav formats. If user wants to create separate folders then those should be created as “01”, “02” and the songs should be with the names 001xxx.mp3/ 002xxx.mp3/ 003xxx.mp3 in those created folders.

MP3 Decoder
pin configuration of SJ2 board to MP3.

Code snippet for MP3

bool mp3__send_command(uint8_t command, uint16_t data) {
  bool status = false;
  uint8_t data_ub = (uint8_t)(data >> 8);
  uint8_t data_lb = (uint8_t)(data);
  mp3_uart_buffer[0] = 0x7e;
  mp3_uart_buffer[1] = 0xff;
  mp3_uart_buffer[2] = 0x06;
  mp3_uart_buffer[3] = command;
  mp3_uart_buffer[4] = 0x00;
  mp3_uart_buffer[5] = data_ub;
  mp3_uart_buffer[6] = data_lb;
  mp3_uart_buffer[7] = 0xef;

  for (int i = 0; i < 8; i++) {
    uart__polled_put(UART__3, mp3_uart_buffer[i]);
  }
  return status = true;
}
MP3 Flow

Software Design

1. Timer - Mode

a) Set the timer to finish the game in given time

b) Arrows change every five seconds


Game rules:

a) Game comprises of 3 levels (level 2 on reaching score 15 and level 3 on reaching score 30).

b) Primary objective is to move hands in direction of arrow if the color is GREEN and in opposite direction if the color is RED.

c) Score increases by 1 point for right hand movement.

d) Level 3 is faster so make sure you tune your reflexes up.

e) Reach score 50 to win the game.


There are 2 tasks involved to ensure functioning of the game.

a) Task 1:

  • Display the game's title screen. The game begins with receiving gesture from any of the player.
  • Also used to display game over screen / win screen.

b) Task 2:

  • Enter into game mode to play the game.
  • Generate arrow matrix frame for random direction(horizontal, vertical, up and down) and random color(Red/Green).
  • Detect the player's gesture from his board, compare it with arrow's location and color.
  • Display score and level.
  • Upon receiving Far gesture from the player, stop game and return to title screen or to restart the game once it is over.

Implementation

Follow Me Game Files

GameCasewithMatrix.png GameCase.png

In the figures above, one SJ2 board will be connected to the PCB to connect to the MP3 decoder, XBEE, and LED matrix and placed inside the 3D printed game case.

PlayerCasewithLid.png PlayerCase.png

In the figures above, another SJ2 board connected to an XBEE will be placed in a player case with the gesture sensor exposed for players to swipe their hands above it.


PackagedUp.jpeg

Testing & Technical Challenges

RGB LED Matrix

1. RGB LED Matrix

  • The ordered matrix was faulty, needed to order another one, project start got delayed a bit due to this.
  • Getting control of the matrix was challenging as it needs to follow specific sequence of pin enabling/disabling at appropriate timing.
  • Setting right delays to avoid flickers and get smooth transitioning between frames by using delays or for loops.
  • Designing elements of the game (title screen and arrow objects) which required careful plotting of pixel data.

Conclusion

It was a really great working experience for all of us. We worked as a team of four which helped us all to know the importance of team-work and co-ordination. We learned many great stuff like

  • State design using FreeRTOS APIs.
  • Uses of static, extern and global variables.
  • Learning optimization through the integration of individual modules
  • Task priorities can effect working of overall project.
  • Workload sharing and cooperation amongst peers.
  • Sharing ideas and debugging a common problem together i.e team building.
  • Hardware and it's design plays important role in development and presentation of the project.

There were a lot of problems while doing this project but it was a steep learning curve. In the end we were able to achieve our objective with really good results.

Project Video

https://youtu.be/N68ExnJIXpk

Project Source Code

  • Git Project Link: [1]
  • Git Project Link(individual modules): [2]

Acknowledgement

We would like to express our gratitude to Professor Preetpal Kang for generously sharing his time and knowledge with us and guiding us through the completion of this project. We would also like to thank our ISA Tirth Pandya for his valuable advice and constructive feedback.

References Used

RGB LED Matrix Interfacing and Designing
MP3 Decoder
Gesture & Orientation Sensors
General/Miscellaneous