Latest revision as of 21:08, 16 December 2022

whack-a-mole

Whack-a-mole

INTO Screen

GAME PLAY Screen1

GAME PLAY Screen2

GAME OVER Screen

Abtract

Whac-a-Mole is a fun arcade game in which the player hits plastic moles with a hammer as soon as it appears out of their hole, to score points. We will be recreating the same old game with some modifications. Instead of plastic moles and a hammer, we will be using a LED Matrix to display mole-like characters and touch sensors to imitate the hammer. The scoring will be based on the no of moles you hit and how accurately you hit. We will be designing three levels and after each level, the difficulty of the game will rise.

Objectives & Introduction

The aim of the game is to hit as many moles as possible as they appear out of the holes

There will be three lives and the speed and complexity of the game increase as the score goes up.

If you miss a mole you lose a life

Based on how many moles you hit your score will increase.

The LED matrix is used to create the 12-mole-hole-like animation and the moles appear randomly in some holes.

ITO-coated PET is used for receiving the touch input (hitting the mole).

A speaker is used to play background music, and sound clips when you hit a mole.

Team Members & Technical Responsibilities

Karthigai Amutha Ezhilarasu
- Touch Sensor Driver Implementation
- Game logic design
- Hardware Integration

Stephen Oneto
- Graphics Driver Implementation
- Game logic design
- Enclosure design and 3D printing
- Battery regulatory circuit design

Yatish Koul
- MP3 Driver Implementation
- Integrating MP3 module with the Game
- Game logic design

Schedule

Week#	Start Date	End Date	Task	Status
1	10/13/2022	10/22/2022	Read Past projects to get some design ideas Finalize our choice of game Game Design Confirm functionalities of the game Decide the parts & material for our project	Completed Completed Completed Completed Completed
2	10/23/2022	10/29/2022	Create a project Wiki page Study previous projects Order parts	Completed Completed Completed
3	10/30/2022	11/5/2022	Create Git repository Test whether all the components are working properly Read and familiarize with the datasheets of the components	Completed Completed Completed
4	10/31/2022	11/12/2022	Draw schematic Design game character Develop graphics driver for LED matrix Develop Driver for audio Develop Driver for touch sensors	Completed Completed Completed Completed Completed
5	11/13/2022	11/19/2022	Test and debug the driver functionality Hardware integration Display game Character on the screen	Completed Completed Completed
6	11/20/2022	11/26/2022	Game score design Integrate game sound with the background music	Completed Completed
7	11/27/2022	12/3/2022	Test & bug fix Design Game enclosure	Completed Completed
8	12/04/2022	12/10/2022	Fine-tuning timing and logic Improve and add animations as necessary Finalizing the video game Update the wiki page.	Completed Completed Completed Completed
9	12/11/2022	12/15/2022	Prepare for the Final Project DEMO Prepare project video Wrap up Wiki page & Project submission Final demo	Completed Completed Completed Completed

Materials List with cost

Part Name	Part Model & Source	Quantity	Cost Per Unit (USD)
Micro-Controller SJ2 Board	SJ2 Board (Purchased from Preet Kang)	1	50.00
RGB LED MATRIX	64X64 RGB - 2.5mm Pitch	1	54.95
Power Supply	Universal AC Adapter	1	11.99
ITO Coated Pet Plastic	ITO (100mm X 200mm)	2	20.00
Touch Sensor Breakout	12-Key Capacitive Touch Sensor Breakout	1	7.95
Speaker	3" Diameter-4Ohm 3 Watt	1	1.95
Audio Decoding Player Module	Hiletgo MP3 Decode Module	1	6.49

Design & Implementation

Schematic

Touch Breakout Board

Hardware Design

The system is built on the SJ2 development board. A 64*64 LED matrix, a touch breakout board, and an MP3 decoder are interfaced with the SJ2 board. The LED matrix screen has a top rectangle to display the score and lives. The remaining portion is divided into 12(4*3) rectangles depicting 12-mole holes. Each of the 12 holes is covered with an ITO-coated flexible transparent sheet. Each of the ITO sheets is connected to the touch sensor breakout board through wires. The system is designed to operate on a battery. The entire system is enclosed in a 3D-printed enclosure.

Hardware Interface

Hardware Design front view

System enclosure

Enclosure Base

Enclosure Lid

Our system enclosure was 3-D printed as two separate pieces. The lid enclosed the LED matrix and provided nice, clean-cut squares separating all of the mole holes. The base was just a simple box with a rectangle cut out of the top allowing for wiring from the SJ2 and power adapter to the LED matrix.

Capacitive Touch Breakout & ITO Sheets

Touch Breakout Board

ITO Sheet

We used a MPR121 12-channel capacitive touch sensor breakout board. we can select one of 4 addresses with the ADDR pin, for a total of 48 capacitive touch pads via I2C. It handles all the filtering and can be configured for different levels of sensitivity. The MPR121 uses a constant DC current capacitance sensing scheme. The voltage measured on the input sensing node is inversely proportional to the capacitance. At the end of each charge circle, this voltage is sampled by an internal 10-bit ADC. The sampled data is then processed through several stages of digital filtering. The digital filtering process allows for good noise immunity in different environments. Once the electrode capacitance data is acquired, the electrode touch/release status is determined by comparing it to the capacitance baseline value. The capacitance baseline is tracked by MPR121 automatically based on the background capacitance variation. The baseline value is compared with the current immediate electrode data to determine if a touch or release has occurred.

We used ITO sheets to pass the finger touch input to the touch breakout board. ITO is Indium oxide doped with tin oxide which is electrical conductivity and optically transparent. The sheet we used is made of flexible PET coated with ITO on one side.

Hardware Design

The ITO sheets are cut into 12 rectangles. They are glued on top of the LED matrix with an approximate gap of one pixel in between to avoid contact. In order to protect the LED matrix from any accidental spoilage from the glue application we used a layer of transparent tape between the LED matrix and the ITO sheets. The 12 ITO buttons are connected to 12 pins of the touch sensor breakout board. When any of the pins detects a touch an interrupt will be generated and given to GPIO P0.1 of the SJ2 board.

Software Design

The interrupt from the touch breakout gives a semaphore from the ISR. On taking the semaphore the code will sequentially scan the status of all 12 pins to identify which pin is being touched. This communication between SJ2 and the touch status registers on the breakout board happens via the I2C bus. If the touched pin has an active mole, then the corresponding mole hit animation happens and a hit sound is played. If the touched pin is not associated with any of the current actives moles, that touch is simply ignored. Since we are ignoring the invalid touches, the touch debounce doesn't cause any problem with the game flow.

Touch Breakout Board

LED Matrix

Hardware Design

The LED matrix used was an Adafruit 64 x 64 matrix with a 2.5mm pitch. This pitch size provided a perfectly-sized screen that was neither too big nor too small. It featured a 14 pin interface (not including the 2 ground pins) which were interfaced to the SJ2 board through GPIO pins. To power the matrix, we repurposed the power cables included in the package and fashioned them to work with a standard 9V AC adapter.

Software Design

API

Software support for the matrix was done entirely through toggling GPIO pins. The most basic level of the API includes functions to set a single pixel, a row of pixels, and the entire board itself. We accomplish this by representing the matrix in memory as a 2D array. The array holds a special 3-bit enumeration type that corresponds to the various RGB values that the matrix is able to visually represent. A standard scan-based matrix lights a certain amount of rows at a time based on the scan rate using shift registers to hold the RGB values of each pixel and address lines to dictate which row(s) to light up. This process is made incredibly easy by simply traversing the 2D array row by row, pixel by pixel and pushing the values at each element to their corresponding RGB GPIO pins. A level above that in the API are the drawing helper functions. These functions provided functionality to draw/set pixels within the 2D array in the form of basic shapes (primarily lines). These functions were instrumental in developing the uppermost layer of our API the sprite layer. This layer uses the drawing helper functions to draw actual objects onto the screen such as alphanumeric characters and game art.

Tasks

From a task level, we had three main display tasks: the screen refresh task, the pattern modification task, and the mole animation task.

Refresh Task

The refresh task simply calls an API function to draw the entire board based on the 2D matrix array in memory, provides semaphores for the other tasks to run, and then sleeps for 8ms. 8ms was chosen through trial and error to give the best game performance to viewing experience ratio. One of the best features of the game is the incredibly smooth animations and this low refresh delay is the key factor in this.

Pattern modification task

The pattern modification task provided a way to make basic modifications to the screen in real-time. Using a global variable, other tasks are able to change the state of the screen by setting the variable to an enumeration value corresponding to a visual pattern. For example, every time a mole is struck the touch detection task sets the current pattern variable to the "Score" enumeration. The modification task picks this up, and increments the score on the screen.

Mole Animation Task

Similar to the modification task, this task performs modifications to the screen, but it is exclusively limited to animation of our mole characters. In essence, it receives a randomly generated 16-bit number from the mole pop task and generates animations for the moles corresponding the bits set in the received number. This is accomplished through a state machine keeping track which state the animation process is currently in at any given time and taking action based on changes in state. An example of this would be when a mole is struck by the player. When a mole is struck, an ISR generated by the touch sensor changes the state member of the structure used to maintain animation data. The next time this task runs, the mole hit state generates a blinking animation and then resets the state to idle.

MP3 Decoder

This module is a straightforward MP3 player that runs on the YX5300 high-quality MP3 audio chip. MP3 and WAV file formats with sampling rates of 8 kHz to 48 kHz are supported. The micro SD card used to store the audio files can be inserted because there is a TF card socket on the board. By sending commands to the module via the UART interface, the MCU can control the MP3 playback state by altering the volume, switching tracks, etc.

Serial MP3 Player

Hardware Design

The UART Tx, UART Rx, VCC, and GND pins are the 4 pins that the MP3 decoder module needs. The SJ2 board provided the module with the 3.3V it needed to start up. On the SJ2 board, UART1 pins have been set up so that commands can be sent to the decoder. This UART communication's baud rate is set at 9600 bps. Additionally, this decoder has a speaker-connected 3.5mm audio jack.

Connection diagram of MP3 decoder to SJ2 board.

Software Design

MP3 Initialization

In our project, the command we use plays the song by navigating to its index on the SD card. In our code, the function that sends this command is referred to as play music at index. We must first send the command [Select device] before we can send the instruction to play music at a specific index. The datasheet specifies that you should send "7E FF 06 09 00 00 02 EF" since Serial MP3 Player only accepts micro SD cards. Additionally, the function uart_polled_put on UART1 is being used to transmit the commands.

Game Logic

Hardware Interface

Task Descriptions

Mole Generation Task

The mole generation task is in charge of randomly generating which mole(s) to pop up next. It is essentially the main "gameplay" driver of our game. It performs this action by generating a random number between 0 and 4096 and sending it over a queue to the mole animation task. We used the number 4096 as the upper limit because that is the highest number that can be represented by 12 bits. These bits correspond to the 12 mole holes that can be triggered. Once this number is generated and sent to the queue, the tasks sleeps for a variable amount of time. The task delay is dependent on the current score that the player has achieved. Using a simple equation, we are able to generate the task delay using the score as input. When the task wakes again, the process repeats.

Screen Refresh Task

The screen refresh task is in charge of translating the data from the 2-D matrix array in memory and pushing it to the LED matrix itself through GPIOs. This task leverages the LED Matrix API that was created (see the LED Matrix section above) to perform a single frame write to the LED matrix. Once it is done producing this frame, it sends a semaphore to both the mole animation task and the screen pattern modification task indicating that the frame was produced and the other tasks are allowed to work on producing the data for the next frame. After this, it sleeps for 8 ticks before starting again and writing the next frame. 8ms was a rather arbitrarily selected number based on gameplay testing. It provided the best balance between performance and smoothness.

Screen Pattern Modification Task

This task was in charge of handling basic screen modification based on a "screen state" variable. The basic loop consists of waiting on a semaphore from the refresh task and modifying the matrix array in memory which the refresh task can represent visually on the physical LED matrix. This modification occurs through pattern handler which takes the current screen state and draws the pattern corresponding to that screen state (see the LED matrix section above).

Mole Animation Task

The mole animation task is in charge of facilitating all of the mole animations going on throughout the LED matrix. The details of this task can be found in the LED Matrix section above, but the basic loop involves waiting on a semaphore from the screen refresh task, receiving a value sent from the mole generation task through a queue (if available) and taking action based on the value received. As mentioned, the mole generation task generates a random 12-bit value that corresponds to which mole holes to trigger animations for. That value is received in the mole animation task and ORd with a global value that tracks the current mole holes with active animations occurring. From there that ORd value is fed into the state machine function which handles each mole hole's animation through independent structures containing state information for each mole hole. This whole loop produces a single frame which the matrix refresh task pushes to the LED matrix and the process start all over again.

Touch Sensor Task

The touch sensor task essentially just waits on a semaphore that is delivered by an ISR that is triggered by the touch sensor breakout board, performs logic to determine which pin was triggered, plays an audio file, and sets various state variables based on that pin. More specifically, when the semaphore is received, the task performs an I2C read to a register on the breakout board dedicated to holding the value of the last touched pin. The task then checks whether this pin corresponds to a mole hole that is currently active. If so, the task modifies members of the mole animation structure that corresponding to the generated pin's mole hole. In addition, it changes the current matrix pattern enumeration value to the Score value which the screen modification task uses to increment the score on the LED screen.

Testing & Technical Challenges

Damaged ITO coating

The most annoying challenge we faced was with the ITO sheets. During the development phase, the ITO buttons we created worked perfectly. As we approach the first prototype demo, after putting all the hardware together we realized that the ITO sheets were not at all working. This happened because we were not careful in handling the ITO sheets. We used tape to keep the wires attached to the sheets. As we fixed and removed the tape so many times to get the perfect connectivity, we damaged the ITO coating. So, we used strips of aluminum foil during the remaining development phase and replaced them with the new ITO sheets just before the final demo. Also, keeping the wires in place is also challenging. So, we designed a 3D-printed enclosure to keep the wires intact and hide the visible wires on the top of the LED matrix.

LED blackout

Another problem we faced during the hardware integration is due to some loose connection of wires. The LED matrix worked fine sometimes but blacked out sometimes. We realized later that this happened because one of the wires was not soldered properly to the PCB.

Touch sensor breakout board short circuit

On the day that the project was due, one of our parts went up in smoke and short circuited to the point where it was unable to be used. This was likely the result of a power line coming into direct contact with a ground contact on the board or vice versa. Our design concept being a smaller, somewhat portable system, we designed our enclosure to have the bare minimum space required to fit all of the components. In doing this, our wiring had to be packed tightly which left plenty of room for error. This component is one of the main components used to drive our system and therefore, it was incredibly hard to simulate gameplay without it for the demo. For future endeavors, we will definitely consider making our packaging less needlessly compacted as well as potentially providing each component its own physical receptacle integrated into the casing, sectioning its contact points off from other components.

Game crash at certain score threshold

We found that the game would crash once the score got to a certain point. The difficulty of the game would scale based on the score of the game; the higher the score, the less delay between random mole generation events and delay of mole being in the up position. Therefore, as the score gets higher, the faster moles are generated and the more touch inputs must be processed. While the exact cause of the crash is still unknown, we compromised on the refresh task delay by a few milliseconds and everything worked perfectly. The CPU usage after lowering this delay was far lower than before and provided much smoother and responsive gameplay.

Mole animation state machine

One of the most challenging aspects of this project was handling the mole animations. Handling 12 separate sections all with independent animations was initially hard to work through conceptually, especially considering that there are multiple viable ways to handle it. For the sake of having a nice compact set of tasks, we went with a state machine-based approach using an array of 12 structures that store state machine data for each "mole hole". This array of structs allowed for totally independent handling of each mole's animations within a single task. Overall, I am very proud of how the animations and the sheer responsiveness of the game turned out as a result of implementing this way.

Conclusion

This project was a valuable teaching tool. Every module used for the project was effectively integrated, and we were able to reproduce our original game concept. During the project, we had a number of challenges, including difficulties starting the LED matrix driver, difficulties figuring out how to integrate touch sensing with so many game objects, difficulties creating and maintaining our complex game logic state machine, and much more. We were able to understand the real-world use of FreeRTOS while working on our project. We came to the conclusion that defining weekly goals and exceeding them could assist compartmentalize the project and eventually meet the desired outcome. This project improved our proficiency with embedded software and solidified our command of it.

Project Video

Whack_a_Mole Demo video

Project Source Code

Whack_a_Mole Git Lab Source Code Link

Acknowledgement

We would like to thank Professor Preetpal Kang for sharing his knowledge and insights with us during the course of the project.

References

List any references used in project.

@@ Line 1: / Line 1: @@
-=== Grading Criteria ===
+[[File:whack-a-mole.gif|right|500px|thumb|whack-a-mole]]
-<font color="green">
+='''Whack-a-mole'''=
-*  How well is Software & Hardware Design described?
+{|
-*  How well can this report be used to reproduce this project?
+|[[File:introscreen.jpg|left|300px|thumb|INTO Screen]]
-*  Code Quality
+|[[File:gamescreen1.jpg|left|300px|thumb|GAME PLAY Screen1]]
-*  Overall Report Quality:
+|[[File:gamescreen2.jpg|left|300px|thumb|GAME PLAY Screen2]]
-**  Software Block Diagrams
+|[[File:gameoverscreen.jpg|left|300px|thumb|GAME OVER Screen]]
-**  Hardware Block Diagrams
+|}
-**:  Schematic Quality
-**  Quality of technical challenges and solutions adopted.
-</font>
-== Project Title ==
+== '''Abtract''' ==
-Whack-a-mole
+Whac-a-Mole is a fun arcade game in which the player hits plastic moles with a hammer as soon as it appears out of their hole, to score points. We will be recreating the same old game with some modifications. Instead of plastic moles and a hammer, we will be using a LED Matrix to display mole-like characters and touch sensors to imitate the hammer. The scoring will be based on the no of moles you hit and how accurately you hit. We will be designing three levels and after each level, the difficulty of the game will rise.
-== Abstract ==
+== '''Objectives & Introduction''' ==
-Whac- a-Mole is a fun arcade game in which the player hits plastic moles with a hammer as soon as it appears out of its hole, to score points. We will be recreating the same old game with some modifications. Instead of plastic moles and a hammer, we will be using a LED Matrix to display mole-like characters and touch sensors to imitate the hammer. The scoring will be based on the no of moles you hit and how accurately you hit. We will be designing three levels and after each level, the difficulty of the game will rise.
-== Objectives & Introduction ==
+*The aim of the game is to hit as many moles as possible as they appear out of the holes
-•	The aim of the game will be to hit as many moles as possible as they appear out of the holes
+*There will be three lives and the speed and complexity of the game increase as the score goes up.
-•	There will be three levels and the speed and complexity of the game increase as the levels go up.
+*If you miss a mole you lose a life
-•	The LED matrix will be used to create the 7 mole-hole-like animation and the moles appear randomly in some holes.
+*Based on how many moles you hit your score will increase.
-•	We are planning to use two different mole reactions, one for appearing from the hole and another for being hit by the player.
+*The LED matrix is used to create the 12-mole-hole-like animation and the moles appear randomly in some holes.
-•	We are planning to use ITO-coated PET for receiving the touch input (hitting the mole).
+*ITO-coated PET is used for receiving the touch input (hitting the mole).
-•	Based on how many moles you hit and accurately you hit your score will increase.
+*A speaker is used to play background music, and sound clips when you hit a mole.
-•	A speaker will be used to play background music, and sound clips when you hit a mole.
+== '''Team Members & Technical Responsibilities''' ==
+*'''''[https://www.linkedin.com/in/karthigai-amutha-ezhilarasu-103a4413a/ Karthigai Amutha Ezhilarasu]'''''
+** <font color="BLACK">Touch Sensor Driver Implementation</font>
+** <font color="BLACK">Game logic design</font>
+** <font color="BLACK">Hardware Integration</font>
-=== Team Members & Responsibilities ===
+*'''''[https://www.linkedin.com/in/stephen-oneto-7836641b9/ Stephen Oneto]'''''
-*  Karthigai Amutha Ezhilarasu
+** <font color="BLACK">Graphics Driver Implementation</font>
+** <font color="BLACK">Game logic design</font>
+** <font color="BLACK">Enclosure design and 3D printing</font>
+** <font color="BLACK">Battery regulatory circuit design</font>
-*  Stephen Oneto
+*'''''[https://www.linkedin.com/in/yatishkoul/?trk=people-guest_people_search-card&originalSubdomain=in/ Yatish Koul]'''''
+** <font color="BLACK">MP3 Driver Implementation</font>
-*  Yatish Koul
+** <font color="BLACK">Integrating MP3 module with the Game</font>
+** <font color="BLACK">Game logic design</font>
-== Schedule ==
+== '''Schedule''' ==
 {| class="wikitable"
@@ Line 53: / Line 57: @@
 ! scope="row"| 1
 |
-* 10/03/2022
+*10/13/2022
 |
-* 10/8/2022
+*10/22/2022
 |
 *Read Past projects to get some design ideas
@@ Line 71: / Line 75: @@
 ! scope="row"| 2
 |
-*10/10/2022
+*10/23/2022
 |
-*10/15/2022
+*10/29/2022
 |
 *Create a project Wiki page
@@ Line 80: / Line 84: @@
 |
 * <span style="color:green">Completed</span>
-* <span style="color:orange">In progress</span>
+* <span style="color:green">Completed</span>
-* <span style="color:orange">In progress</span>
+* <span style="color:green">Completed</span>
 |-
 ! scope="row"| 3
 |
-*10/17/2022
+*10/30/2022
 |
-*10/22/2022
+*11/5/2022
 |
-*Draw schematic
 *Create Git repository
 *Test whether all the components are working properly
 *Read and familiarize with the datasheets of the components
 |
-* <span style="color:red">Not started</span>
+* <span style="color:green">Completed</span>
-* <span style="color:red">Not started</span>
+* <span style="color:green">Completed</span>
-* <span style="color:red">Not started</span>
+* <span style="color:green">Completed</span>
-* <span style="color:red">Not started</span>
 |-
 ! scope="row"| 4
@@ Line 105: / Line 105: @@
 * 10/31/2022
 |
-* 11/05/2022
+* 11/12/2022
 |
+*Draw schematic
 *Design game character
 *Develop graphics driver for LED matrix
-*Develop Driver for audio.
+*Develop Driver for audio
-*Test and debug the driver functionality.
+*Develop Driver for touch sensors
 |
-* <span style="color:red">Not started</span>
+* <span style="color:green">Completed</span>
-* <span style="color:red">Not started</span>
+* <span style="color:green">Completed</span>
-* <span style="color:red">Not started</span>
+* <span style="color:green">Completed</span>
-* <span style="color:red">Not started</span>
+* <span style="color:green">Completed</span>
+* <span style="color:green">Completed</span>
 |-
 ! scope="row"| 5
 |
-*11/07/2022
+*11/13/2022
 |
-*11/12/2022
+*11/19/2022
 |
-* Integrate game sound with the background music
+*Test and debug the driver functionality
-* Display game Character on the screen
+*Hardware integration
-* Game score design
+*Display game Character on the screen
 |
-* <span style="color:red">Not started</span>
+* <span style="color:green">Completed</span>
-* <span style="color:red">Not started</span>
+* <span style="color:green">Completed</span>
-* <span style="color:red">Not started</span>
+* <span style="color:green">Completed</span>
 |-
 ! scope="row"| 6
 |
-*11/14/2022
+*11/20/2022
 |
-*11/19/2022
+*11/26/2022
 |
-*Integrate Game logic with music and animations.
-*Design menu screens, song selection
+*Game score design
-*Design PCB
+*Integrate game sound with the background music
-*Design Game enclosure.
 |
-* <span style="color:red">Not started</span>
+* <span style="color:green">Completed</span>
-* <span style="color:red">Not started</span>
+* <span style="color:green">Completed</span>
-* <span style="color:red">Not started</span>
-* <span style="color:red">Not started</span>
 |-
 ! scope="row"| 7
 |
-* 11/21/2022
+* 11/27/2022
 |
-* 11/26/2022
+* 12/3/2022
 |
 *Test & bug fix
-*Fine tuning timing and logic
+*Design Game enclosure
-*Improve and add animations as necessary
 |
-* <span style="color:red">Not started</span>
+* <span style="color:green">Completed</span>
-* <span style="color:red">Not started</span>
+* <span style="color:green">Completed</span>
-* <span style="color:red">Not started</span>
 |-
 ! scope="row"| 8
 |
-* 11/28/2022
+* 12/04/2022
 |
-* 12/03/2022
+* 12/10/2022
 |
-* Finalizing the video game
+*Fine-tuning timing and logic
-* Update the wiki page.
+*Improve and add animations as necessary
+*Finalizing the video game
+*Update the wiki page.
 |
-* <span style="color:red">Not started</span>
+* <span style="color:green">Completed</span>
-* <span style="color:red">Not started</span>
+* <span style="color:green">Completed</span>
-* <span style="color:red">Not started</span>
+* <span style="color:green">Completed</span>
+* <span style="color:green">Completed</span>
 |-
 ! scope="row"| 9
 |
-* 12/05/2022
+* 12/11/2022
 |
-* 12/10/2022
+* 12/15/2022
 |
 *Prepare for the Final Project DEMO
+*Prepare project video
 *Wrap up Wiki page & Project submission
+*Final demo
 |
-* <span style="color:red">Not started</span>
+* <span style="color:green">Completed</span>
-* <span style="color:red">Not started</span>
+* <span style="color:green">Completed</span>
+* <span style="color:green">Completed</span>
+* <span style="color:green">Completed</span>
+|}
+== '''Materials List with cost''' ==
+{| class="wikitable"
+|-
+! scope="col" | Part Name
+! scope="col" | Part Model & Source
+! scope="col" | Quantity
+! scope="col" | Cost Per Unit (USD)
+|-
+|
+* Micro-Controller SJ2 Board
+|
+* SJ2 Board (Purchased from Preet Kang)
+|
+* 1
+|
+* 50.00
 |-
-! scope="row"| 10
+|
+* RGB LED MATRIX
 |
-*12/11/2022
+* 64X64 RGB - 2.5mm Pitch
 |
-*12/11/2022
+* 1
+|
+* 54.95
+|-
 |
-*Final Demo
+* Power Supply
-*Update the wiki page(if needed)
+|
+* Universal AC Adapter
+|
+* 1
+|
+* 11.99
+|-
 |
-* <span style="color:red">Not started</span>
+* ITO Coated Pet Plastic
-* <span style="color:red">Not started</span>
+|
+* ITO (100mm X 200mm)
+|
+* 2
+|
+* 20.00
 |-
+|
+* Touch Sensor Breakout
+|
+* 12-Key Capacitive Touch Sensor Breakout
+|
+* 1
+|
+* 7.95
+|-
+|
+* Speaker
+|
+* 3" Diameter-4Ohm 3 Watt
+|
+* 1
+|
+* 1.95
+|-
+|
+* Audio Decoding Player Module
+|
+* Hiletgo MP3 Decode Module
+|
+* 1
+|
+* 6.49
 |}
-<HR>
+== '''Design & Implementation''' ==
-<BR/>
+=== '''Schematic''' ===
-== Parts List & Cost ==
+{|
-Give a simple list of the cost of your project broken down by components.  Do not write long stories here.
+|[[File:Schematic_whack_a_mole.png|thumb|left|1000px|Touch Breakout Board]]
+|}
-== Design & Implementation ==
+=== '''Hardware Design''' ===
-The design section can go over your hardware and software design.  Organize this section using sub-sections that go over your design and implementation.
-=== Hardware Design ===
+The system is built on the SJ2 development board. A 64*64 LED matrix, a touch breakout board, and an MP3 decoder are interfaced with the SJ2 board. The LED matrix screen has a top rectangle to display the score and lives. The remaining portion is divided into 12(4*3) rectangles depicting 12-mole holes. Each of the 12 holes is covered with an ITO-coated flexible transparent sheet. Each of the ITO sheets is connected to the touch sensor breakout board through wires. The system is designed to operate on a battery. The entire system is enclosed in a 3D-printed enclosure.
-Discuss your hardware design here.  Show detailed schematics, and the interface here.
+{|
+|[[File:hardware_interface.png|thumb|center|850px|Hardware Interface]]
+|[[File:basic_hardware_design.jpg|thumb|center|400px|Hardware Design front view]]
+|}
-=== Hardware Interface ===
+=== '''System enclosure''' ===
-In this section, you can describe how your hardware communicates, such as which BUSes used.  You can discuss your driver implementation here, such that the '''Software Design''' section is isolated to talk about high level workings rather than inner working of your project.
+{|
+[[File:Enclosure base.png|300px|thumb|left|Enclosure Base]]
+|[[File:Enclosure lid.png|300px|thumb|left|Enclosure Lid]]
+|}
-=== Software Design ===
+Our system enclosure was 3-D printed as two separate pieces. The lid enclosed the LED matrix and provided nice, clean-cut squares separating all of the mole holes. The base was just a simple box with a rectangle cut out of the top allowing for wiring from the SJ2 and power adapter to the LED matrix.
-Show your software design.  For example, if you are designing an MP3 Player, show the tasks that you are using, and what they are doing at a high level.  Do not show the details of the code.  For example, do not show exact code, but you may show psuedocode and fragments of code.  Keep in mind that you are showing DESIGN of your software, not the inner workings of it.
-=== Implementation ===
+=== '''Capacitive Touch Breakout & ITO Sheets'''===
-This section includes implementation, but again, not the details, just the high level.  For example, you can list the steps it takes to communicate over a sensor, or the steps needed to write a page of memory onto SPI Flash.  You can include sub-sections for each of your component implementation.
-== Testing & Technical Challenges ==
+{|
-Describe the challenges of your project.  What advise would you give yourself or someone else if your project can be started from scratch again?
+|[[File:touch_breakout_board.jpg|thumb|center|300px|Touch Breakout Board]]
-Make a smooth transition to testing section and described what it took to test your project.
+|[[File:ITO sheet.jpg|thumb|center|300px|ITO Sheet]]
+|}
+We used a [https://www.adafruit.com/product/1982?gclid=CjwKCAiAs8acBhA1EiwAgRFdwzB3af9LPV_44DPSljvhmlTxMCtEujRYov8PMwWzihz4s6SwvxKhfRoCJp8QAvD_BwE/ MPR121] 12-channel capacitive touch sensor breakout board. we can select one of 4 addresses with the ADDR pin, for a total of 48 capacitive touch pads via I2C. It handles all the filtering and can be configured for different levels of sensitivity. The MPR121 uses a constant DC current capacitance sensing scheme. The voltage measured on the input sensing node is inversely proportional to the capacitance. At the end of each charge circle, this voltage is sampled by an internal 10-bit ADC. The sampled data is then processed through several stages of digital filtering. The digital filtering process allows for good noise immunity in different environments. Once the electrode capacitance data is acquired, the electrode touch/release status is determined by comparing it to the capacitance baseline value. The capacitance baseline is tracked by MPR121 automatically based on the background capacitance variation. The baseline value is compared with the current immediate electrode data to determine if a touch or release has occurred.
-Include sub-sections that list out a problem and solution, such as:
+We used ITO sheets to pass the finger touch input to the touch breakout board. ITO is Indium oxide doped with tin oxide which is electrical conductivity and optically transparent. The sheet we used is made of flexible PET coated with ITO on one side.
-=== <Bug/issue name> ===
+=====<u> '''Hardware Design'''</u> =====
-Discuss the issue and resolution.
-== Conclusion ==
+The ITO sheets are cut into 12 rectangles. They are glued on top of the LED matrix with an approximate gap of one pixel in between to avoid contact. In order to protect the LED matrix from any accidental spoilage from the glue application we used a layer of transparent tape between the LED matrix and the ITO sheets. The 12 ITO buttons are connected to 12 pins of the touch sensor breakout board. When any of the pins detects a touch an interrupt will be generated and given to GPIO P0.1 of the SJ2 board.
-Conclude your project here.  You can recap your testing and problems.  You should address the "so what" part here to indicate what you ultimately learnt from this project.  How has this project increased your knowledge?
-=== Project Video ===
+===== <u>'''Software Design''' </u>=====
-Upload a video of your project and post the link here.
-=== Project Source Code ===
+The interrupt from the touch breakout gives a semaphore from the ISR. On taking the semaphore the code will sequentially scan the status of all 12 pins to identify which pin is being touched. This communication between SJ2 and the touch status registers on the breakout board happens via the I2C bus. If the touched pin has an active mole, then the corresponding mole hit animation happens and a hit sound is played. If the touched pin is not associated with any of the current actives moles, that touch is simply ignored. Since we are ignoring the invalid touches, the touch debounce doesn't cause any problem with the game flow.
-*  [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]
-== References ==
+{|
-=== Acknowledgement ===
+|[[File:touch_flowchart.png|thumb|center|300px|Touch Breakout Board]]
-Any acknowledgement that you may wish to provide can be included here.
+|}
-=== References Used ===
+==='''LED Matrix'''===
+{|
+|[[File:LED_matrix.jpg|thumb|left|300px|LED Matrix]]
+|}
+=====<u> '''Hardware Design'''</u> =====
+The LED matrix used was an Adafruit 64 x 64 matrix with a 2.5mm pitch. This pitch size provided a perfectly-sized screen that was neither too big nor too small. It featured a 14 pin interface (not including the 2 ground pins) which were interfaced to the SJ2 board through GPIO pins. To power the matrix, we repurposed the power cables included in the package and fashioned them to work with a standard 9V AC adapter.
+===== <u>'''Software Design''' </u>=====
+====== '''API''' ======
+Software support for the matrix was done entirely through toggling GPIO pins. The most basic level of the API includes functions to set a single pixel, a row of pixels, and the entire board itself. We accomplish this by representing the matrix in memory as a 2D array. The array holds a special 3-bit  enumeration type that corresponds to the various RGB values that the matrix is able to visually represent. A standard scan-based matrix lights a certain amount of rows at a time based on the scan rate using shift registers to hold the RGB values of each pixel and address lines to dictate which row(s) to light up. This process is made incredibly easy by simply traversing the 2D array row by row, pixel by pixel and pushing the values at each element to their corresponding RGB GPIO pins. A level above that in the API are the drawing helper functions. These functions provided functionality to draw/set pixels within the 2D array in the form of basic shapes (primarily lines). These functions were instrumental in developing the uppermost layer of our API the sprite layer. This layer uses the drawing helper functions to draw actual objects onto the screen such as alphanumeric characters and game art.
+====== '''Tasks''' ======
+From a task level, we had three main display tasks: the screen refresh task, the pattern modification task, and the mole animation task.
+====== Refresh Task ======
+The refresh task simply calls an API function to draw the entire board based on the 2D matrix array in memory, provides semaphores for the other tasks to run, and then sleeps for 8ms. 8ms was chosen through trial and error to give the best game performance to viewing experience ratio. One of the best features of the game is the incredibly smooth animations and this low refresh delay is the key factor in this.
+====== Pattern modification task ======
+The pattern modification task provided a way to make basic modifications to the screen in real-time. Using a global variable, other tasks are able to change the state of the screen by setting the variable to an enumeration value corresponding to a visual pattern. For example, every time a mole is struck the touch detection task sets the current pattern variable to the "Score" enumeration. The modification task picks this up, and increments the score on the screen.
+====== Mole Animation Task ======
+Similar to the modification task, this task performs modifications to the screen, but it is exclusively limited to animation of our mole characters. In essence, it receives a randomly generated 16-bit number from the mole pop task and generates animations for the moles corresponding the bits set in the received number. This is accomplished through a state machine keeping track which state the animation process is currently in at any given time and taking action based on changes in state. An example of this would be when a mole is struck by the player. When a mole is struck, an ISR generated by the touch sensor changes the state member of the structure used to maintain animation data. The next time this task runs, the mole hit state generates a blinking animation and then resets the state to idle.
+=== '''MP3 Decoder''' ===
+This module is a straightforward MP3 player that runs on the YX5300 high-quality MP3 audio chip. MP3 and WAV file formats with sampling rates of 8 kHz to 48 kHz are supported. The micro SD card used to store the audio files can be inserted because there is a TF card socket on the board. By sending commands to the module via the UART interface, the MCU can control the MP3 playback state by altering the volume, switching tracks, etc.
+{|
+|[[File:MP3_Decoder_board.JPG|300px|thumb|left|Serial MP3 Player]]
+|}
+=====<u> '''Hardware Design'''</u> =====
+The UART Tx, UART Rx, VCC, and GND pins are the 4 pins that the MP3 decoder module needs. The SJ2 board provided the module with the 3.3V it needed to start up. On the SJ2 board, UART1 pins have been set up so that commands can be sent to the decoder. This UART communication's baud rate is set at 9600 bps. Additionally, this decoder has a speaker-connected 3.5mm audio jack.
+{|
+|[[File:MP3_Decoder_SJ2_Connection.JPG|700px|thumb|left|Connection diagram of MP3 decoder to SJ2 board.]]
+|}
+===== <u>'''Software Design''' </u>=====
+{|
+|[[File:MP3_Initialization.png|700px|thumb|center|MP3 Initialization]]
+|}
+In our project, the command we use plays the song by navigating to its index on the SD card. In our code, the function that sends this command is referred to as play music at index. We must first send the command [Select device] before we can send the instruction to play music at a specific index. The datasheet specifies that you should send "7E FF 06 09 00 00 02 EF" since Serial MP3 Player only accepts micro SD cards. Additionally, the function uart_polled_put on UART1 is being used to transmit the commands.
+== '''Game Logic'''==
+{|
+|[[File:Classic flow chart.jpg|thumb|left|800px|Hardware Interface]]
+|}
+=== '''Task Descriptions''' ===
+[[File:Tasks_whack_a_mole.jpg]]
+==== '''Mole Generation Task''' ====
+The mole generation task is in charge of randomly generating which mole(s) to pop up next. It is essentially the main "gameplay" driver of our game. It performs this action by generating a random number between 0 and 4096 and sending it over a queue to the mole animation task. We used the number 4096 as the upper limit because that is the highest number that can be represented by 12 bits. These bits correspond to the 12 mole holes that can be triggered. Once this number is generated and sent to the queue, the tasks sleeps for a variable amount of time. The task delay is dependent on the current score that the player has achieved. Using a simple equation, we are able to generate the task delay using the score as input. When the task wakes again, the process repeats.
+==== '''Screen Refresh Task''' ====
+The screen refresh task is in charge of translating the data from the 2-D matrix array in memory and pushing it to the LED matrix itself through GPIOs. This task leverages the LED Matrix API that was created (see the LED Matrix section above) to perform a single frame write to the LED matrix. Once it is done producing this frame, it sends a semaphore to both the mole animation task and the screen pattern modification task indicating that the frame was produced and the other tasks are allowed to work on producing the data for the next frame. After this, it sleeps for 8 ticks before starting again and writing the next frame. 8ms was a rather arbitrarily selected number based on gameplay testing. It provided the best balance between performance and smoothness.
+==== '''Screen Pattern Modification Task''' ====
+This task was in charge of handling basic screen modification based on a "screen state" variable. The basic loop consists of waiting on a semaphore from the refresh task and modifying the matrix array in memory which the refresh task can represent visually on the physical LED matrix. This modification occurs through pattern handler which takes the current screen state and draws the pattern corresponding to that screen state (see the LED matrix section above).
+==== '''Mole Animation Task''' ====
+The mole animation task is in charge of facilitating all of the mole animations going on throughout the LED matrix. The details of this task can be found in the LED Matrix section above, but the basic loop involves waiting on a semaphore from the screen refresh task, receiving a value sent from the mole generation task through a queue (if available) and taking action based on the value received. As mentioned, the mole generation task generates a random 12-bit value that corresponds to which mole holes to trigger animations for. That value is received in the mole animation task and ORd with a global value that tracks the current mole holes with active animations occurring. From there that ORd value is fed into the state machine function which handles each mole hole's animation through independent structures containing state information for each mole hole. This whole loop produces a single frame which the matrix refresh task pushes to the LED matrix and the process start all over again.
+==== '''Touch Sensor Task''' ====
+The touch sensor task essentially just waits on a semaphore that is delivered by an ISR that is triggered by the touch sensor breakout board, performs logic to determine which pin was triggered, plays an audio file, and sets various state variables based on that pin. More specifically, when the semaphore is received, the task performs an I2C read to a register on the breakout board dedicated to holding the value of the last touched pin. The task then checks whether this pin corresponds to a mole hole that is currently active. If so, the task modifies members of the mole animation structure that corresponding to the generated pin's mole hole. In addition, it changes the current matrix pattern enumeration value to the Score value which the screen modification task uses to increment the score on the LED screen.
+== '''Testing & Technical Challenges''' ==
+=== '''Damaged ITO coating''' ===
+The most annoying challenge we faced was with the ITO sheets. During the development phase, the ITO buttons we created worked perfectly. As we approach the first prototype demo, after putting all the hardware together we realized that the ITO sheets were not at all working. This happened because we were not careful in handling the ITO sheets. We used tape to keep the wires attached to the sheets. As we fixed and removed the tape so many times to get the perfect connectivity, we damaged the ITO coating. So, we used strips of aluminum foil during the remaining development phase and replaced them with the new ITO sheets just before the final demo. Also, keeping the wires in place is also challenging. So, we designed a 3D-printed enclosure to keep the wires intact and hide the visible wires on the top of the LED matrix.
+=== '''LED blackout''' ===
+Another problem we faced during the hardware integration is due to some loose connection of wires. The LED matrix worked fine sometimes but blacked out sometimes. We realized later that this happened because one of the wires was not soldered properly to the PCB.
+=== '''Touch sensor breakout board short circuit''' ===
+On the day that the project was due, one of our parts went up in smoke and short circuited to the point where it was unable to be used. This was likely the result of a power line coming into direct contact with a ground contact on the board or vice versa. Our design concept being a smaller, somewhat portable system, we designed our enclosure to have the bare minimum space required to fit all of the components. In doing this, our wiring had to be packed tightly which left plenty of room for error. This component is one of the main components used to drive our system and therefore, it was incredibly hard to simulate gameplay without it for the demo. For future endeavors, we will definitely consider making our packaging less needlessly compacted as well as potentially providing each component its own physical receptacle integrated into the casing, sectioning its contact points off from other components.
+=== '''Game crash at certain score threshold''' ===
+We found that the game would crash once the score got to a certain point. The difficulty of the game would scale based on the score of the game; the higher the score, the less delay between random mole generation events and delay of mole being in the up position. Therefore, as the score gets higher, the faster moles are generated and the more touch inputs must be processed. While the exact cause of the crash is still unknown, we compromised on the refresh task delay by a few milliseconds and everything worked perfectly. The CPU usage after lowering this delay was far lower than before and provided much smoother and responsive gameplay.
+=== '''Mole animation state machine''' ===
+One of the most challenging aspects of this project was handling the mole animations. Handling 12 separate sections all with independent animations was initially hard to work through conceptually, especially considering that there are multiple viable ways to handle it. For the sake of having a nice compact set of tasks, we went with a state machine-based approach using an array of 12 structures that store state machine data for each "mole hole". This array of structs allowed for totally independent handling of each mole's animations within a single task. Overall, I am very proud of how the animations and the sheer responsiveness of the game  turned out as a result of implementing this way.
+== '''Conclusion''' ==
+This project was a valuable teaching tool. Every module used for the project was effectively integrated, and we were able to reproduce our original game concept. During the project, we had a number of challenges, including difficulties starting the LED matrix driver, difficulties figuring out how to integrate touch sensing with so many game objects, difficulties creating and maintaining our complex game logic state machine, and much more. We were able to understand the real-world use of FreeRTOS while working on our project. We came to the conclusion that defining weekly goals and exceeding them could assist compartmentalize the project and eventually meet the desired outcome. This project improved our proficiency with embedded software and solidified our command of it.
+== '''Project Video''' ==
+[https://www.youtube.com/watch?v=ZjppX6ZrMRQ&t=199s&ab_channel=KarthigaiAmutha  Whack_a_Mole Demo video]
+== '''Project Source Code''' ==
+[https://gitlab.com/karthigaiamutha.ezhilarasu/whack_a_mole/  Whack_a_Mole Git Lab Source Code Link]
+== '''Acknowledgement''' ==
+We would like to thank Professor Preetpal Kang for sharing his knowledge and insights with us during the course of the project.
+== '''References''' ==
 List any references used in project.
+# [http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems CMPE244 Bookstack]
-=== Appendix ===
+# [https://cdn-shop.adafruit.com/datasheets/MPR121.pdf Touch Breakout Board Datasheet]
-You can list the references you used.
+# [http://geekmatic.in.ua/pdf/Catalex_MP3_board.pdf Serial MP3 Player Datasheet]
+# [https://www.classicgame.com/game/Whack+a+Mole  Whack_a_Mole Miniclip Game]

Difference between revisions of "F22: xGameCreate"

Latest revision as of 21:08, 16 December 2022

Contents

Whack-a-mole

Abtract

Objectives & Introduction

Team Members & Technical Responsibilities

Schedule

Materials List with cost

Design & Implementation

Schematic

Hardware Design

System enclosure

Capacitive Touch Breakout & ITO Sheets

Hardware Design

Software Design

LED Matrix

Hardware Design

Software Design

API

Tasks

Refresh Task

Pattern modification task

Mole Animation Task

MP3 Decoder

Hardware Design

Software Design

Game Logic

Task Descriptions

Mole Generation Task

Screen Refresh Task

Screen Pattern Modification Task

Mole Animation Task

Touch Sensor Task

Testing & Technical Challenges

Damaged ITO coating

LED blackout

Touch sensor breakout board short circuit

Game crash at certain score threshold

Mole animation state machine

Conclusion

Project Video

Project Source Code

Acknowledgement

References

Navigation menu

Search