Embedded Systems Learning Academy - User contributions [en]

F24: Jardinains

2024-12-19T23:32:20Z

Proj user1: /* Implementation */

=== Grading Criteria ===

* How well is Software & Hardware Design described?
* How well can this report be used to reproduce this project?
* Code Quality
* Overall Report Quality:
** Software Block Diagrams
** Hardware Block Diagrams
**: Schematic Quality
** Quality of technical challenges and solutions adopted.


== Project Title ==
Jardinaiins

== Abstract ==
Prepare for endless bouncing, breaking, and gnome-busting excitement! Jardinaiins reinvents classic brick-breaking gameplay with a delightful twist. Dive into vibrant, garden-themed levels filled with challenging obstacles, quirky gnome antics, and thrilling power-ups. Smash through barriers, outsmart mischievous gnomes, and uncover surprises as you progress in this fun and addictive adventure!

== Objectives & Introduction ==
• Interface the SJ2 board with the Waveshare 64x64 RGB LED Matrix Panel to display dynamic text, colorful images, and animations. The onboard GPIOs of the SJ2 microcontroller will drive the LED panel, and its chainable design will be utilized to facilitate scalability for larger displays.
• Use the MMA845XQ accelerometer onboard the SJ2 board as the player controller, mapping tilt and motion inputs to control animations, select media, or interact with displayed content on the LED matrix.
• Integrate the HiLetgo GD3300 UART Control Serial MP3 Music Player Module to enable digital audio output by leveraging the SJ2 board’s UART protocol. The system will send serial commands for operations like play, pause, track selection, and volume control.
• Utilize the General Purpose Direct Memory Access (GPDMA) controller on the SJ2 board for memory-to-memory transfers, reducing delays during video buffer updates for the LED matrix.
• Leverage the DMA controller's memory-to-peripheral function to minimize frequent I2S interrupts while loading digital audio content to the MP3 module, optimizing system performance.
• Implement an SD card peripheral driver on the SJ2 board to facilitate reading and writing of multimedia data, enabling seamless storage and retrieval of audio and visual content for playback and display.

=== Team Members & Responsibilities ===
* Dev Hitesh Malkan
** Core gameplay development
** Gitlab and integrating all parts of code
* Nidhi Sharma
** Interface LED matrix with SJ2 board and core gameplay
** Ensure coding standards
* Sunny Shah
** Audio system research and design
** Assist on all parts of the project
* Navya
** MP3 module driver development
** Interfacing the accelerometer

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
* 10/14/2024
* 10/14/2024
|
* 10/20/2024
* 10/20/2024
|
* Review past projects, gather insights, and discuss with team members
* Initial ordering of parts (SJ2 board, 64x64 LED Matrix, GD3300 MP3 module).
|
* Completed
* Completed
|-
! scope="row"| 2
|
* 10/21/2024
* 10/21/2024
|
* 10/27/2024
* 10/27/2024
|
* Submit project proposals.
* Identify and select sensors and peripherals.
|
* Completed
* Completed
|-
! scope="row"| 3
|
* 10/28/2024
* 10/28/2024
|
* 11/03/2024
* 11/03/2024
|
* Learn about LED matrices and display "Jardinains."
* Study accelerometer principles and operation.
|
* Completed
* Completed
|-
! scope="row"| 4
|
* 11/04/2024
* 11/04/2024
* 11/04/2024
* 11/04/2024
|
* 11/10/2024
* 11/10/2024
* 11/10/2024
* 11/10/2024
|
* Create gitlab repo for the project.
* Develop basic game graphics and user interfaces on LED matrix.
* Develop code to control LED matrix based on accelerometer input.
* Familiarize with GD3300 MP3 decoder module.
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
* 11/11/2024
* 11/11/2024
* 11/11/2024
|
* 11/17/2024
* 11/17/2024
* 11/17/2024
|
* Design and implement game rules and systems.
* Integrate game logic with LED matrix display.
* Integrate sound effects with game logic.
|
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
* 11/18/2024
* 11/18/2024
* 11/18/2024
|
* 11/24/2024
* 11/24/2024
* 11/24/2024
|
* Test hardware integration and debug accelerometer-controlled input.
* Optimize DMA usage for efficient LED matrix updates.
* Validate UART-based communication with GD3300 MP3 module.
|
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/25/2024
* 11/25/2024
* 11/25/2024
|
* 12/01/2024
* 12/01/2024
* 12/01/2024
|
* Implement advanced game mechanics (e.g., power-ups, increasing difficulty).
* Optimize and fine-tune user experience and interactions.
* Test and validate system performance under real-time conditions.
|
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 12/02/2024
* 12/02/2024
* 12/02/2024
|
* 12/08/2024
* 12/08/2024
* 12/08/2024
|
* Finalize sprite designs and integrate animations with LED matrix.
* Conduct system-wide integration testing.
* Address edge cases in game logic and audio synchronization.
|
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 12/09/2024
* 12/09/2024
* 12/09/2024
|
* 12/17/2024
* 12/17/2024
* 12/17/2024
|
* Perform final testing of the complete system, ensuring stability and performance.
* Prepare and conduct final demonstrations.
* Submit final project code to GitLab and update project documentation and wiki.
|
* Completed
* Completed
* Completed
|-
|}

<HR>
 

== Parts List & Cost ==
{| class="wikitable"
|-
! scope="col"| #
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| SJ2 Board
| 1
| $50.00
|-
! scope="row"| 2
| Waveshare 64x64 4096 Pixels RGB Full-Color LED Matrix Panel
| Amazon
| 1
| $43.00
|-
! scope="row"| 3
| HiLetgo GD3300
| Amazon
| 1
| $8.00
|-
! scope="row"| 4
| SPST Normal Open ON/Off 2 Pin Round Button
| Amazon
| 3
| $8.00
|-
! scope="row"| 5
| MMA845XQ
| N/A
| 1
| N/A
|-
! scope="row"| 6
| SD card
| N/A
| 1
| N/A
|-
|}
<HR>
 

== Design & Implementation ==
The design section can go over your hardware and software design. Organize this section using sub-sections that go over your design and implementation.

=== Hardware Design ===
Discuss your hardware design here. Show detailed schematics, and the interface here.

=== Hardware Interface ===
In this section, you can describe how your hardware communicates, such as which BUSes used. You can discuss your driver implementation here, such that the '''Software Design''' section is isolated to talk about high level workings rather than inner working of your project.

=== Software Design ===
Controller
The accelerometer on the SJ2 board manages paddle movement by sampling the y-axis magnitude 17 times over a 17-millisecond period and calculating the average. This value is capped at a maximum of 700 within a range of 1000. To make it simpler, the tilt value is divided by 31, producing 32 possible readings, which correspond to 5 bits of data. Each time there is a change in magnitude or a button press, the controller transmits a single byte of data to the console. This byte encapsulates all the necessary information for controlling the on-screen paddle. The first two bits indicate button input, with 0 representing no input and 1 indicating input detection. The third bit specifies the tilt direction, where 0 means left and 1 means right. The remaining five bits represent the accelerometer's magnitude.

Audio
Sound effects are added using MP3 Serial Decoder HiLetgo GD3300, source code was written on our own.

=== Implementation ===
Paddle Movement :

The paddle movement is driven by accelerometer data wirelessly transmitted from the SJ-Two board to the LPC4088 board. This data is mapped to 63 possible positions for the paddle. Initially, the paddle would instantly move to the position corresponding to the accelerometer input. However, this approach was refined by introducing an algorithm that allows for a smoother transition between positions. The algorithm calculates paddle velocity based on the "current desired position" and the previous position. Parameters of the algorithm were fine-tuned iteratively to achieve a smooth and responsive paddle movement.

Paddle-Ball Collision Logic :

The ball's position is updated each frame by adding delta_x and delta_y to its current x and y coordinates. If the ball's next y-position crosses the paddle's y-position, a paddle collision check is performed. To do this, the ball's movement is modeled as a slope derived from delta_x and delta_y. Using this slope, along with the ball's current position and one pixel above the paddle's y-position, a point-slope equation (y2−y1)=m(x2−x1)(y_2 - y_1) = m(x_2 - x_1)(y2−y1)=m(x2−x1) is used to calculate the x-position of the ball at the time it reaches that y-coordinate.
Once the expected collision point is determined, the ball's collision magnitude and its next position's magnitude are calculated. By comparing these magnitudes, a time fraction for the frame is derived to estimate when the ball might have collided with the paddle. Using the paddle's current position and velocity, the paddle's previous position is estimated to determine how far it moved during this time fraction. By multiplying the time fraction by the paddle's x-position change, it is possible to check whether the paddle was underneath the ball at the moment it reached the pixel above the paddle's y-position.

== Testing & Technical Challenges ==
Ball-Collision Logic
The initial ball collision logic relied on pixel values to detect when the ball entered a brick. However, this approach failed to handle cases where the ball entered a brick diagonally, requiring the logic to be completely reworked. In the updated version, the focus shifted to determining whether the ball was crossing into an area containing a brick. This was achieved by dividing the playing field into a grid representing the positions of the bricks.
The new logic involved an initial check to see if the ball was crossing a boundary near a brick. If the ball crossed only the x-boundary, a collision check was performed to determine if it hit a brick. Similarly, if it crossed only the y-boundary, another collision check was conducted. In cases where the ball crossed both x- and y-boundaries, additional logic was applied to determine which boundary the ball crossed first. This updated method effectively handled diagonal collisions and improved the overall accuracy of the collision detection.
This experience highlighted the value of abandoning ineffective solutions and adopting new approaches, as they can lead to better outcomes.
Paddle Movement
We encountered an issue where the paddle would slow down as it neared the walls. This happened because the difference between the "current desired paddle position" and the previous paddle position decreased near the walls, resulting in a reduced paddle velocity. To address this, we adjusted the logic so that whenever the accelerometer data mapped to the minimum or maximum paddle index, the "current desired paddle position" was extended. This adjustment ensured that the paddle velocity remained consistent, preventing the paddle from slowing down as it approached the walls.
Paddle – Collision Logic
During final integration and testing, one major issue we faced was a bug where the ball failed to detect collisions with the paddle, causing it to pass through instead of bouncing off. To troubleshoot this, we implemented a clever debugging method: changing the ball's colour during each collision check. This allowed us to visually confirm whether the collision logic was being executed. Through this approach, we discovered that the section of code responsible for checking paddle collisions was being skipped, even when the ball's next y-position had crossed the paddle's y-position. This debugging method proved invaluable in identifying and resolving the root cause of the issue.

== Conclusion ==
We thoroughly enjoyed the process of creating this project and gained a wealth of knowledge along the way. Our driver development skills were sharpened as we wrote drivers for various modules, including the LED matrix and MP3 decoder. Additionally, our understanding of the FreeRTOS API deepened, and we acquired fundamental game and graphics development skills through our work with the LED matrix.
Throughout the project, we faced numerous challenges, such as getting the LED matrix driver to function properly, implementing collision detection with multiple game objects, and designing and maintaining a complex game logic state machine, among many others. Each obstacle provided valuable learning experiences and contributed to our growth as developers. By applying design principles and communication protocols learned in lectures, we successfully brought this project to life. Key concepts we utilized include UART, SPI, I2C, I2S, DMA, FreeRTOS mutexes, binary semaphores, queues, task creation, stack sizing, task scheduling, interrupt service routines, and bootloaders.
Through countless hours of testing and verification, we learned the critical importance of allocating sufficient time for integration and debugging, as it greatly increases the likelihood of a successful outcome. Without this phase, the project might have faced significant issues upon delivery. We hope this project serves as a helpful resource for future engineers and wish you all great success and enjoyable development experiences ahead!

=== Project Video ===
Upload a video of your project and post the link here.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
We extend our heartfelt gratitude to Professor Preet and the ISA for organizing such an outstanding class and setting high standards. Their guidance and expectations continually motivated us to strive for excellence and surpass our own limits.

=== References Used ===
List any references used in project.

=== Appendix ===
You can list the references you used.

F24: Jardinains

2024-12-19T23:31:54Z

Proj user1: /* Software Design */

=== Grading Criteria ===

* How well is Software & Hardware Design described?
* How well can this report be used to reproduce this project?
* Code Quality
* Overall Report Quality:
** Software Block Diagrams
** Hardware Block Diagrams
**: Schematic Quality
** Quality of technical challenges and solutions adopted.


== Project Title ==
Jardinaiins

== Abstract ==
Prepare for endless bouncing, breaking, and gnome-busting excitement! Jardinaiins reinvents classic brick-breaking gameplay with a delightful twist. Dive into vibrant, garden-themed levels filled with challenging obstacles, quirky gnome antics, and thrilling power-ups. Smash through barriers, outsmart mischievous gnomes, and uncover surprises as you progress in this fun and addictive adventure!

== Objectives & Introduction ==
• Interface the SJ2 board with the Waveshare 64x64 RGB LED Matrix Panel to display dynamic text, colorful images, and animations. The onboard GPIOs of the SJ2 microcontroller will drive the LED panel, and its chainable design will be utilized to facilitate scalability for larger displays.
• Use the MMA845XQ accelerometer onboard the SJ2 board as the player controller, mapping tilt and motion inputs to control animations, select media, or interact with displayed content on the LED matrix.
• Integrate the HiLetgo GD3300 UART Control Serial MP3 Music Player Module to enable digital audio output by leveraging the SJ2 board’s UART protocol. The system will send serial commands for operations like play, pause, track selection, and volume control.
• Utilize the General Purpose Direct Memory Access (GPDMA) controller on the SJ2 board for memory-to-memory transfers, reducing delays during video buffer updates for the LED matrix.
• Leverage the DMA controller's memory-to-peripheral function to minimize frequent I2S interrupts while loading digital audio content to the MP3 module, optimizing system performance.
• Implement an SD card peripheral driver on the SJ2 board to facilitate reading and writing of multimedia data, enabling seamless storage and retrieval of audio and visual content for playback and display.

=== Team Members & Responsibilities ===
* Dev Hitesh Malkan
** Core gameplay development
** Gitlab and integrating all parts of code
* Nidhi Sharma
** Interface LED matrix with SJ2 board and core gameplay
** Ensure coding standards
* Sunny Shah
** Audio system research and design
** Assist on all parts of the project
* Navya
** MP3 module driver development
** Interfacing the accelerometer

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
* 10/14/2024
* 10/14/2024
|
* 10/20/2024
* 10/20/2024
|
* Review past projects, gather insights, and discuss with team members
* Initial ordering of parts (SJ2 board, 64x64 LED Matrix, GD3300 MP3 module).
|
* Completed
* Completed
|-
! scope="row"| 2
|
* 10/21/2024
* 10/21/2024
|
* 10/27/2024
* 10/27/2024
|
* Submit project proposals.
* Identify and select sensors and peripherals.
|
* Completed
* Completed
|-
! scope="row"| 3
|
* 10/28/2024
* 10/28/2024
|
* 11/03/2024
* 11/03/2024
|
* Learn about LED matrices and display "Jardinains."
* Study accelerometer principles and operation.
|
* Completed
* Completed
|-
! scope="row"| 4
|
* 11/04/2024
* 11/04/2024
* 11/04/2024
* 11/04/2024
|
* 11/10/2024
* 11/10/2024
* 11/10/2024
* 11/10/2024
|
* Create gitlab repo for the project.
* Develop basic game graphics and user interfaces on LED matrix.
* Develop code to control LED matrix based on accelerometer input.
* Familiarize with GD3300 MP3 decoder module.
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
* 11/11/2024
* 11/11/2024
* 11/11/2024
|
* 11/17/2024
* 11/17/2024
* 11/17/2024
|
* Design and implement game rules and systems.
* Integrate game logic with LED matrix display.
* Integrate sound effects with game logic.
|
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
* 11/18/2024
* 11/18/2024
* 11/18/2024
|
* 11/24/2024
* 11/24/2024
* 11/24/2024
|
* Test hardware integration and debug accelerometer-controlled input.
* Optimize DMA usage for efficient LED matrix updates.
* Validate UART-based communication with GD3300 MP3 module.
|
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/25/2024
* 11/25/2024
* 11/25/2024
|
* 12/01/2024
* 12/01/2024
* 12/01/2024
|
* Implement advanced game mechanics (e.g., power-ups, increasing difficulty).
* Optimize and fine-tune user experience and interactions.
* Test and validate system performance under real-time conditions.
|
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 12/02/2024
* 12/02/2024
* 12/02/2024
|
* 12/08/2024
* 12/08/2024
* 12/08/2024
|
* Finalize sprite designs and integrate animations with LED matrix.
* Conduct system-wide integration testing.
* Address edge cases in game logic and audio synchronization.
|
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 12/09/2024
* 12/09/2024
* 12/09/2024
|
* 12/17/2024
* 12/17/2024
* 12/17/2024
|
* Perform final testing of the complete system, ensuring stability and performance.
* Prepare and conduct final demonstrations.
* Submit final project code to GitLab and update project documentation and wiki.
|
* Completed
* Completed
* Completed
|-
|}

<HR>
 

== Parts List & Cost ==
{| class="wikitable"
|-
! scope="col"| #
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| SJ2 Board
| 1
| $50.00
|-
! scope="row"| 2
| Waveshare 64x64 4096 Pixels RGB Full-Color LED Matrix Panel
| Amazon
| 1
| $43.00
|-
! scope="row"| 3
| HiLetgo GD3300
| Amazon
| 1
| $8.00
|-
! scope="row"| 4
| SPST Normal Open ON/Off 2 Pin Round Button
| Amazon
| 3
| $8.00
|-
! scope="row"| 5
| MMA845XQ
| N/A
| 1
| N/A
|-
! scope="row"| 6
| SD card
| N/A
| 1
| N/A
|-
|}
<HR>
 

== Design & Implementation ==
The design section can go over your hardware and software design. Organize this section using sub-sections that go over your design and implementation.

=== Hardware Design ===
Discuss your hardware design here. Show detailed schematics, and the interface here.

=== Hardware Interface ===
In this section, you can describe how your hardware communicates, such as which BUSes used. You can discuss your driver implementation here, such that the '''Software Design''' section is isolated to talk about high level workings rather than inner working of your project.

=== Software Design ===
Controller
The accelerometer on the SJ2 board manages paddle movement by sampling the y-axis magnitude 17 times over a 17-millisecond period and calculating the average. This value is capped at a maximum of 700 within a range of 1000. To make it simpler, the tilt value is divided by 31, producing 32 possible readings, which correspond to 5 bits of data. Each time there is a change in magnitude or a button press, the controller transmits a single byte of data to the console. This byte encapsulates all the necessary information for controlling the on-screen paddle. The first two bits indicate button input, with 0 representing no input and 1 indicating input detection. The third bit specifies the tilt direction, where 0 means left and 1 means right. The remaining five bits represent the accelerometer's magnitude.

Audio
Sound effects are added using MP3 Serial Decoder HiLetgo GD3300, source code was written on our own.

=== Implementation ===
Paddle Movement
The paddle movement is driven by accelerometer data wirelessly transmitted from the SJ-Two board to the LPC4088 board. This data is mapped to 63 possible positions for the paddle. Initially, the paddle would instantly move to the position corresponding to the accelerometer input. However, this approach was refined by introducing an algorithm that allows for a smoother transition between positions. The algorithm calculates paddle velocity based on the "current desired position" and the previous position. Parameters of the algorithm were fine-tuned iteratively to achieve a smooth and responsive paddle movement.
Paddle-Ball Collision Logic
The ball's position is updated each frame by adding delta_x and delta_y to its current x and y coordinates. If the ball's next y-position crosses the paddle's y-position, a paddle collision check is performed. To do this, the ball's movement is modeled as a slope derived from delta_x and delta_y. Using this slope, along with the ball's current position and one pixel above the paddle's y-position, a point-slope equation (y2−y1)=m(x2−x1)(y_2 - y_1) = m(x_2 - x_1)(y2−y1)=m(x2−x1) is used to calculate the x-position of the ball at the time it reaches that y-coordinate.
Once the expected collision point is determined, the ball's collision magnitude and its next position's magnitude are calculated. By comparing these magnitudes, a time fraction for the frame is derived to estimate when the ball might have collided with the paddle. Using the paddle's current position and velocity, the paddle's previous position is estimated to determine how far it moved during this time fraction. By multiplying the time fraction by the paddle's x-position change, it is possible to check whether the paddle was underneath the ball at the moment it reached the pixel above the paddle's y-position.

== Testing & Technical Challenges ==
Ball-Collision Logic
The initial ball collision logic relied on pixel values to detect when the ball entered a brick. However, this approach failed to handle cases where the ball entered a brick diagonally, requiring the logic to be completely reworked. In the updated version, the focus shifted to determining whether the ball was crossing into an area containing a brick. This was achieved by dividing the playing field into a grid representing the positions of the bricks.
The new logic involved an initial check to see if the ball was crossing a boundary near a brick. If the ball crossed only the x-boundary, a collision check was performed to determine if it hit a brick. Similarly, if it crossed only the y-boundary, another collision check was conducted. In cases where the ball crossed both x- and y-boundaries, additional logic was applied to determine which boundary the ball crossed first. This updated method effectively handled diagonal collisions and improved the overall accuracy of the collision detection.
This experience highlighted the value of abandoning ineffective solutions and adopting new approaches, as they can lead to better outcomes.
Paddle Movement
We encountered an issue where the paddle would slow down as it neared the walls. This happened because the difference between the "current desired paddle position" and the previous paddle position decreased near the walls, resulting in a reduced paddle velocity. To address this, we adjusted the logic so that whenever the accelerometer data mapped to the minimum or maximum paddle index, the "current desired paddle position" was extended. This adjustment ensured that the paddle velocity remained consistent, preventing the paddle from slowing down as it approached the walls.
Paddle – Collision Logic
During final integration and testing, one major issue we faced was a bug where the ball failed to detect collisions with the paddle, causing it to pass through instead of bouncing off. To troubleshoot this, we implemented a clever debugging method: changing the ball's colour during each collision check. This allowed us to visually confirm whether the collision logic was being executed. Through this approach, we discovered that the section of code responsible for checking paddle collisions was being skipped, even when the ball's next y-position had crossed the paddle's y-position. This debugging method proved invaluable in identifying and resolving the root cause of the issue.

== Conclusion ==
We thoroughly enjoyed the process of creating this project and gained a wealth of knowledge along the way. Our driver development skills were sharpened as we wrote drivers for various modules, including the LED matrix and MP3 decoder. Additionally, our understanding of the FreeRTOS API deepened, and we acquired fundamental game and graphics development skills through our work with the LED matrix.
Throughout the project, we faced numerous challenges, such as getting the LED matrix driver to function properly, implementing collision detection with multiple game objects, and designing and maintaining a complex game logic state machine, among many others. Each obstacle provided valuable learning experiences and contributed to our growth as developers. By applying design principles and communication protocols learned in lectures, we successfully brought this project to life. Key concepts we utilized include UART, SPI, I2C, I2S, DMA, FreeRTOS mutexes, binary semaphores, queues, task creation, stack sizing, task scheduling, interrupt service routines, and bootloaders.
Through countless hours of testing and verification, we learned the critical importance of allocating sufficient time for integration and debugging, as it greatly increases the likelihood of a successful outcome. Without this phase, the project might have faced significant issues upon delivery. We hope this project serves as a helpful resource for future engineers and wish you all great success and enjoyable development experiences ahead!

=== Project Video ===
Upload a video of your project and post the link here.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
We extend our heartfelt gratitude to Professor Preet and the ISA for organizing such an outstanding class and setting high standards. Their guidance and expectations continually motivated us to strive for excellence and surpass our own limits.

=== References Used ===
List any references used in project.

=== Appendix ===
You can list the references you used.

F24: Jardinains

2024-12-19T23:31:25Z

Proj user1: /* */

=== Grading Criteria ===

* How well is Software & Hardware Design described?
* How well can this report be used to reproduce this project?
* Code Quality
* Overall Report Quality:
** Software Block Diagrams
** Hardware Block Diagrams
**: Schematic Quality
** Quality of technical challenges and solutions adopted.


== Project Title ==
Jardinaiins

== Abstract ==
Prepare for endless bouncing, breaking, and gnome-busting excitement! Jardinaiins reinvents classic brick-breaking gameplay with a delightful twist. Dive into vibrant, garden-themed levels filled with challenging obstacles, quirky gnome antics, and thrilling power-ups. Smash through barriers, outsmart mischievous gnomes, and uncover surprises as you progress in this fun and addictive adventure!

== Objectives & Introduction ==
• Interface the SJ2 board with the Waveshare 64x64 RGB LED Matrix Panel to display dynamic text, colorful images, and animations. The onboard GPIOs of the SJ2 microcontroller will drive the LED panel, and its chainable design will be utilized to facilitate scalability for larger displays.
• Use the MMA845XQ accelerometer onboard the SJ2 board as the player controller, mapping tilt and motion inputs to control animations, select media, or interact with displayed content on the LED matrix.
• Integrate the HiLetgo GD3300 UART Control Serial MP3 Music Player Module to enable digital audio output by leveraging the SJ2 board’s UART protocol. The system will send serial commands for operations like play, pause, track selection, and volume control.
• Utilize the General Purpose Direct Memory Access (GPDMA) controller on the SJ2 board for memory-to-memory transfers, reducing delays during video buffer updates for the LED matrix.
• Leverage the DMA controller's memory-to-peripheral function to minimize frequent I2S interrupts while loading digital audio content to the MP3 module, optimizing system performance.
• Implement an SD card peripheral driver on the SJ2 board to facilitate reading and writing of multimedia data, enabling seamless storage and retrieval of audio and visual content for playback and display.

=== Team Members & Responsibilities ===
* Dev Hitesh Malkan
** Core gameplay development
** Gitlab and integrating all parts of code
* Nidhi Sharma
** Interface LED matrix with SJ2 board and core gameplay
** Ensure coding standards
* Sunny Shah
** Audio system research and design
** Assist on all parts of the project
* Navya
** MP3 module driver development
** Interfacing the accelerometer

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
* 10/14/2024
* 10/14/2024
|
* 10/20/2024
* 10/20/2024
|
* Review past projects, gather insights, and discuss with team members
* Initial ordering of parts (SJ2 board, 64x64 LED Matrix, GD3300 MP3 module).
|
* Completed
* Completed
|-
! scope="row"| 2
|
* 10/21/2024
* 10/21/2024
|
* 10/27/2024
* 10/27/2024
|
* Submit project proposals.
* Identify and select sensors and peripherals.
|
* Completed
* Completed
|-
! scope="row"| 3
|
* 10/28/2024
* 10/28/2024
|
* 11/03/2024
* 11/03/2024
|
* Learn about LED matrices and display "Jardinains."
* Study accelerometer principles and operation.
|
* Completed
* Completed
|-
! scope="row"| 4
|
* 11/04/2024
* 11/04/2024
* 11/04/2024
* 11/04/2024
|
* 11/10/2024
* 11/10/2024
* 11/10/2024
* 11/10/2024
|
* Create gitlab repo for the project.
* Develop basic game graphics and user interfaces on LED matrix.
* Develop code to control LED matrix based on accelerometer input.
* Familiarize with GD3300 MP3 decoder module.
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
* 11/11/2024
* 11/11/2024
* 11/11/2024
|
* 11/17/2024
* 11/17/2024
* 11/17/2024
|
* Design and implement game rules and systems.
* Integrate game logic with LED matrix display.
* Integrate sound effects with game logic.
|
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
* 11/18/2024
* 11/18/2024
* 11/18/2024
|
* 11/24/2024
* 11/24/2024
* 11/24/2024
|
* Test hardware integration and debug accelerometer-controlled input.
* Optimize DMA usage for efficient LED matrix updates.
* Validate UART-based communication with GD3300 MP3 module.
|
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/25/2024
* 11/25/2024
* 11/25/2024
|
* 12/01/2024
* 12/01/2024
* 12/01/2024
|
* Implement advanced game mechanics (e.g., power-ups, increasing difficulty).
* Optimize and fine-tune user experience and interactions.
* Test and validate system performance under real-time conditions.
|
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 12/02/2024
* 12/02/2024
* 12/02/2024
|
* 12/08/2024
* 12/08/2024
* 12/08/2024
|
* Finalize sprite designs and integrate animations with LED matrix.
* Conduct system-wide integration testing.
* Address edge cases in game logic and audio synchronization.
|
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 12/09/2024
* 12/09/2024
* 12/09/2024
|
* 12/17/2024
* 12/17/2024
* 12/17/2024
|
* Perform final testing of the complete system, ensuring stability and performance.
* Prepare and conduct final demonstrations.
* Submit final project code to GitLab and update project documentation and wiki.
|
* Completed
* Completed
* Completed
|-
|}

<HR>
 

== Parts List & Cost ==
{| class="wikitable"
|-
! scope="col"| #
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| SJ2 Board
| 1
| $50.00
|-
! scope="row"| 2
| Waveshare 64x64 4096 Pixels RGB Full-Color LED Matrix Panel
| Amazon
| 1
| $43.00
|-
! scope="row"| 3
| HiLetgo GD3300
| Amazon
| 1
| $8.00
|-
! scope="row"| 4
| SPST Normal Open ON/Off 2 Pin Round Button
| Amazon
| 3
| $8.00
|-
! scope="row"| 5
| MMA845XQ
| N/A
| 1
| N/A
|-
! scope="row"| 6
| SD card
| N/A
| 1
| N/A
|-
|}
<HR>
 

== Design & Implementation ==
The design section can go over your hardware and software design. Organize this section using sub-sections that go over your design and implementation.

=== Hardware Design ===
Discuss your hardware design here. Show detailed schematics, and the interface here.

=== Hardware Interface ===
In this section, you can describe how your hardware communicates, such as which BUSes used. You can discuss your driver implementation here, such that the '''Software Design''' section is isolated to talk about high level workings rather than inner working of your project.

=== Software Design ===
Controller
The accelerometer on the SJ2 board manages paddle movement by sampling the y-axis magnitude 17 times over a 17-millisecond period and calculating the average. This value is capped at a maximum of 700 within a range of 1000. To make it simpler, the tilt value is divided by 31, producing 32 possible readings, which correspond to 5 bits of data. Each time there is a change in magnitude or a button press, the controller transmits a single byte of data to the console. This byte encapsulates all the necessary information for controlling the on-screen paddle. The first two bits indicate button input, with 0 representing no input and 1 indicating input detection. The third bit specifies the tilt direction, where 0 means left and 1 means right. The remaining five bits represent the accelerometer's magnitude.

Audio
Sound effects are added using MP3 Serial Decoder HiLetgo GD3300, source code was written on our own.

=== Implementation ===
Paddle Movement
The paddle movement is driven by accelerometer data wirelessly transmitted from the SJ-Two board to the LPC4088 board. This data is mapped to 63 possible positions for the paddle. Initially, the paddle would instantly move to the position corresponding to the accelerometer input. However, this approach was refined by introducing an algorithm that allows for a smoother transition between positions. The algorithm calculates paddle velocity based on the "current desired position" and the previous position. Parameters of the algorithm were fine-tuned iteratively to achieve a smooth and responsive paddle movement.
Paddle-Ball Collision Logic
The ball's position is updated each frame by adding delta_x and delta_y to its current x and y coordinates. If the ball's next y-position crosses the paddle's y-position, a paddle collision check is performed. To do this, the ball's movement is modeled as a slope derived from delta_x and delta_y. Using this slope, along with the ball's current position and one pixel above the paddle's y-position, a point-slope equation (y2−y1)=m(x2−x1)(y_2 - y_1) = m(x_2 - x_1)(y2−y1)=m(x2−x1) is used to calculate the x-position of the ball at the time it reaches that y-coordinate.
Once the expected collision point is determined, the ball's collision magnitude and its next position's magnitude are calculated. By comparing these magnitudes, a time fraction for the frame is derived to estimate when the ball might have collided with the paddle. Using the paddle's current position and velocity, the paddle's previous position is estimated to determine how far it moved during this time fraction. By multiplying the time fraction by the paddle's x-position change, it is possible to check whether the paddle was underneath the ball at the moment it reached the pixel above the paddle's y-position.

== Testing & Technical Challenges ==
Ball-Collision Logic
The initial ball collision logic relied on pixel values to detect when the ball entered a brick. However, this approach failed to handle cases where the ball entered a brick diagonally, requiring the logic to be completely reworked. In the updated version, the focus shifted to determining whether the ball was crossing into an area containing a brick. This was achieved by dividing the playing field into a grid representing the positions of the bricks.
The new logic involved an initial check to see if the ball was crossing a boundary near a brick. If the ball crossed only the x-boundary, a collision check was performed to determine if it hit a brick. Similarly, if it crossed only the y-boundary, another collision check was conducted. In cases where the ball crossed both x- and y-boundaries, additional logic was applied to determine which boundary the ball crossed first. This updated method effectively handled diagonal collisions and improved the overall accuracy of the collision detection.
This experience highlighted the value of abandoning ineffective solutions and adopting new approaches, as they can lead to better outcomes.
Paddle Movement
We encountered an issue where the paddle would slow down as it neared the walls. This happened because the difference between the "current desired paddle position" and the previous paddle position decreased near the walls, resulting in a reduced paddle velocity. To address this, we adjusted the logic so that whenever the accelerometer data mapped to the minimum or maximum paddle index, the "current desired paddle position" was extended. This adjustment ensured that the paddle velocity remained consistent, preventing the paddle from slowing down as it approached the walls.
Paddle – Collision Logic
During final integration and testing, one major issue we faced was a bug where the ball failed to detect collisions with the paddle, causing it to pass through instead of bouncing off. To troubleshoot this, we implemented a clever debugging method: changing the ball's colour during each collision check. This allowed us to visually confirm whether the collision logic was being executed. Through this approach, we discovered that the section of code responsible for checking paddle collisions was being skipped, even when the ball's next y-position had crossed the paddle's y-position. This debugging method proved invaluable in identifying and resolving the root cause of the issue.

== Conclusion ==
We thoroughly enjoyed the process of creating this project and gained a wealth of knowledge along the way. Our driver development skills were sharpened as we wrote drivers for various modules, including the LED matrix and MP3 decoder. Additionally, our understanding of the FreeRTOS API deepened, and we acquired fundamental game and graphics development skills through our work with the LED matrix.
Throughout the project, we faced numerous challenges, such as getting the LED matrix driver to function properly, implementing collision detection with multiple game objects, and designing and maintaining a complex game logic state machine, among many others. Each obstacle provided valuable learning experiences and contributed to our growth as developers. By applying design principles and communication protocols learned in lectures, we successfully brought this project to life. Key concepts we utilized include UART, SPI, I2C, I2S, DMA, FreeRTOS mutexes, binary semaphores, queues, task creation, stack sizing, task scheduling, interrupt service routines, and bootloaders.
Through countless hours of testing and verification, we learned the critical importance of allocating sufficient time for integration and debugging, as it greatly increases the likelihood of a successful outcome. Without this phase, the project might have faced significant issues upon delivery. We hope this project serves as a helpful resource for future engineers and wish you all great success and enjoyable development experiences ahead!

=== Project Video ===
Upload a video of your project and post the link here.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
We extend our heartfelt gratitude to Professor Preet and the ISA for organizing such an outstanding class and setting high standards. Their guidance and expectations continually motivated us to strive for excellence and surpass our own limits.

=== References Used ===
List any references used in project.

=== Appendix ===
You can list the references you used.

F24: Jardinains

2024-12-19T23:30:31Z

Proj user1: /* Team Members & Responsibilities */

=== Grading Criteria ===

* How well is Software & Hardware Design described?
* How well can this report be used to reproduce this project?
* Code Quality
* Overall Report Quality:
** Software Block Diagrams
** Hardware Block Diagrams
**: Schematic Quality
** Quality of technical challenges and solutions adopted.


== Project Title ==
Jardinaiins

== Abstract ==
Prepare for endless bouncing, breaking, and gnome-busting excitement! Jardinaiins reinvents classic brick-breaking gameplay with a delightful twist. Dive into vibrant, garden-themed levels filled with challenging obstacles, quirky gnome antics, and thrilling power-ups. Smash through barriers, outsmart mischievous gnomes, and uncover surprises as you progress in this fun and addictive adventure!

== Objectives & Introduction ==
• Interface the SJ2 board with the Waveshare 64x64 RGB LED Matrix Panel to display dynamic text, colorful images, and animations. The onboard GPIOs of the SJ2 microcontroller will drive the LED panel, and its chainable design will be utilized to facilitate scalability for larger displays.
• Use the MMA845XQ accelerometer onboard the SJ2 board as the player controller, mapping tilt and motion inputs to control animations, select media, or interact with displayed content on the LED matrix.
• Integrate the HiLetgo GD3300 UART Control Serial MP3 Music Player Module to enable digital audio output by leveraging the SJ2 board’s UART protocol. The system will send serial commands for operations like play, pause, track selection, and volume control.
• Utilize the General Purpose Direct Memory Access (GPDMA) controller on the SJ2 board for memory-to-memory transfers, reducing delays during video buffer updates for the LED matrix.
• Leverage the DMA controller's memory-to-peripheral function to minimize frequent I2S interrupts while loading digital audio content to the MP3 module, optimizing system performance.
• Implement an SD card peripheral driver on the SJ2 board to facilitate reading and writing of multimedia data, enabling seamless storage and retrieval of audio and visual content for playback and display.

=== Team Members & Responsibilities ===
* Dev Hitesh Malkan
** Core gameplay development
** Gitlab and integrating all parts of code
* Nidhi Sharma
** Interface LED matrix with SJ2 board and core gameplay
** Ensure coding standards
* Sunny Shah
** Audio system research and design
** Assist on all parts of the project
* Navya
** MP3 module driver development
** Interfacing the accelerometer

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
* 10/14/2024
* 10/14/2024
|
* 10/20/2024
* 10/20/2024
|
* Review past projects, gather insights, and discuss with team members
* Initial ordering of parts (SJ2 board, 64x64 LED Matrix, GD3300 MP3 module).
|
* Completed
* Completed
|-
! scope="row"| 2
|
* 10/21/2024
* 10/21/2024
|
* 10/27/2024
* 10/27/2024
|
* Submit project proposals.
* Identify and select sensors and peripherals.
|
* Completed
* Completed
|-
! scope="row"| 3
|
* 10/28/2024
* 10/28/2024
|
* 11/03/2024
* 11/03/2024
|
* Learn about LED matrices and display "Jardinains."
* Study accelerometer principles and operation.
|
* Completed
* Completed
|-
! scope="row"| 4
|
* 11/04/2024
* 11/04/2024
* 11/04/2024
* 11/04/2024
|
* 11/10/2024
* 11/10/2024
* 11/10/2024
* 11/10/2024
|
* Create gitlab repo for the project.
* Develop basic game graphics and user interfaces on LED matrix.
* Develop code to control LED matrix based on accelerometer input.
* Familiarize with GD3300 MP3 decoder module.
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
* 11/11/2024
* 11/11/2024
* 11/11/2024
|
* 11/17/2024
* 11/17/2024
* 11/17/2024
|
* Design and implement game rules and systems.
* Integrate game logic with LED matrix display.
* Integrate sound effects with game logic.
|
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
* 11/18/2024
* 11/18/2024
* 11/18/2024
|
* 11/24/2024
* 11/24/2024
* 11/24/2024
|
* Test hardware integration and debug accelerometer-controlled input.
* Optimize DMA usage for efficient LED matrix updates.
* Validate UART-based communication with GD3300 MP3 module.
|
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/25/2024
* 11/25/2024
* 11/25/2024
|
* 12/01/2024
* 12/01/2024
* 12/01/2024
|
* Implement advanced game mechanics (e.g., power-ups, increasing difficulty).
* Optimize and fine-tune user experience and interactions.
* Test and validate system performance under real-time conditions.
|
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 12/02/2024
* 12/02/2024
* 12/02/2024
|
* 12/08/2024
* 12/08/2024
* 12/08/2024
|
* Finalize sprite designs and integrate animations with LED matrix.
* Conduct system-wide integration testing.
* Address edge cases in game logic and audio synchronization.
|
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 12/09/2024
* 12/09/2024
* 12/09/2024
|
* 12/17/2024
* 12/17/2024
* 12/17/2024
|
* Perform final testing of the complete system, ensuring stability and performance.
* Prepare and conduct final demonstrations.
* Submit final project code to GitLab and update project documentation and wiki.
|
* Completed
* Completed
* Completed
|-
|}

<HR>
 

== Parts List & Cost ==
{| class="wikitable"
|-
! scope="col"| #
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| SJ2 Board
| 1
| $50.00
|-
! scope="row"| 2
| Waveshare 64x64 4096 Pixels RGB Full-Color LED Matrix Panel
| Amazon
| 1
| $43.00
|-
! scope="row"| 3
| HiLetgo GD3300
| Amazon
| 1
| $8.00
|-
! scope="row"| 4
| SPST Normal Open ON/Off 2 Pin Round Button
| Amazon
| 3
| $8.00
|-
! scope="row"| 5
| MMA845XQ
| N/A
| 1
| N/A
|-
! scope="row"| 6
| SD card
| N/A
| 1
| N/A
|-
|}
<HR>
 

== Design & Implementation ==
The design section can go over your hardware and software design. Organize this section using sub-sections that go over your design and implementation.

=== Hardware Design ===
Discuss your hardware design here. Show detailed schematics, and the interface here.

=== Hardware Interface ===
In this section, you can describe how your hardware communicates, such as which BUSes used. You can discuss your driver implementation here, such that the '''Software Design''' section is isolated to talk about high level workings rather than inner working of your project.

=== Software Design ===
Controller
The accelerometer on the SJ2 board manages paddle movement by sampling the y-axis magnitude 17 times over a 17-millisecond period and calculating the average. This value is capped at a maximum of 700 within a range of 1000. To make it simpler, the tilt value is divided by 31, producing 32 possible readings, which correspond to 5 bits of data. Each time there is a change in magnitude or a button press, the controller transmits a single byte of data to the console. This byte encapsulates all the necessary information for controlling the on-screen paddle. The first two bits indicate button input, with 0 representing no input and 1 indicating input detection. The third bit specifies the tilt direction, where 0 means left and 1 means right. The remaining five bits represent the accelerometer's magnitude.

Audio
Sound effects are added using MP3 Serial Decoder HiLetgo GD3300, source code was written on our own.

=== Implementation ===
Paddle Movement
The paddle movement is driven by accelerometer data wirelessly transmitted from the SJ-Two board to the LPC4088 board. This data is mapped to 63 possible positions for the paddle. Initially, the paddle would instantly move to the position corresponding to the accelerometer input. However, this approach was refined by introducing an algorithm that allows for a smoother transition between positions. The algorithm calculates paddle velocity based on the "current desired position" and the previous position. Parameters of the algorithm were fine-tuned iteratively to achieve a smooth and responsive paddle movement.
Paddle-Ball Collision Logic
The ball's position is updated each frame by adding delta_x and delta_y to its current x and y coordinates. If the ball's next y-position crosses the paddle's y-position, a paddle collision check is performed. To do this, the ball's movement is modeled as a slope derived from delta_x and delta_y. Using this slope, along with the ball's current position and one pixel above the paddle's y-position, a point-slope equation (y2−y1)=m(x2−x1)(y_2 - y_1) = m(x_2 - x_1)(y2−y1)=m(x2−x1) is used to calculate the x-position of the ball at the time it reaches that y-coordinate.
Once the expected collision point is determined, the ball's collision magnitude and its next position's magnitude are calculated. By comparing these magnitudes, a time fraction for the frame is derived to estimate when the ball might have collided with the paddle. Using the paddle's current position and velocity, the paddle's previous position is estimated to determine how far it moved during this time fraction. By multiplying the time fraction by the paddle's x-position change, it is possible to check whether the paddle was underneath the ball at the moment it reached the pixel above the paddle's y-position.

== Testing & Technical Challenges ==
Ball-Collision Logic
The initial ball collision logic relied on pixel values to detect when the ball entered a brick. However, this approach failed to handle cases where the ball entered a brick diagonally, requiring the logic to be completely reworked. In the updated version, the focus shifted to determining whether the ball was crossing into an area containing a brick. This was achieved by dividing the playing field into a grid representing the positions of the bricks.
The new logic involved an initial check to see if the ball was crossing a boundary near a brick. If the ball crossed only the x-boundary, a collision check was performed to determine if it hit a brick. Similarly, if it crossed only the y-boundary, another collision check was conducted. In cases where the ball crossed both x- and y-boundaries, additional logic was applied to determine which boundary the ball crossed first. This updated method effectively handled diagonal collisions and improved the overall accuracy of the collision detection.
This experience highlighted the value of abandoning ineffective solutions and adopting new approaches, as they can lead to better outcomes.
Paddle Movement
We encountered an issue where the paddle would slow down as it neared the walls. This happened because the difference between the "current desired paddle position" and the previous paddle position decreased near the walls, resulting in a reduced paddle velocity. To address this, we adjusted the logic so that whenever the accelerometer data mapped to the minimum or maximum paddle index, the "current desired paddle position" was extended. This adjustment ensured that the paddle velocity remained consistent, preventing the paddle from slowing down as it approached the walls.
Paddle – Collision Logic
During final integration and testing, one major issue we faced was a bug where the ball failed to detect collisions with the paddle, causing it to pass through instead of bouncing off. To troubleshoot this, we implemented a clever debugging method: changing the ball's colour during each collision check. This allowed us to visually confirm whether the collision logic was being executed. Through this approach, we discovered that the section of code responsible for checking paddle collisions was being skipped, even when the ball's next y-position had crossed the paddle's y-position. This debugging method proved invaluable in identifying and resolving the root cause of the issue.

=== <Bug/issue name> ===
Discuss the issue and resolution.

== Conclusion ==
We thoroughly enjoyed the process of creating this project and gained a wealth of knowledge along the way. Our driver development skills were sharpened as we wrote drivers for various modules, including the LED matrix and MP3 decoder. Additionally, our understanding of the FreeRTOS API deepened, and we acquired fundamental game and graphics development skills through our work with the LED matrix.
Throughout the project, we faced numerous challenges, such as getting the LED matrix driver to function properly, implementing collision detection with multiple game objects, and designing and maintaining a complex game logic state machine, among many others. Each obstacle provided valuable learning experiences and contributed to our growth as developers. By applying design principles and communication protocols learned in lectures, we successfully brought this project to life. Key concepts we utilized include UART, SPI, I2C, I2S, DMA, FreeRTOS mutexes, binary semaphores, queues, task creation, stack sizing, task scheduling, interrupt service routines, and bootloaders.
Through countless hours of testing and verification, we learned the critical importance of allocating sufficient time for integration and debugging, as it greatly increases the likelihood of a successful outcome. Without this phase, the project might have faced significant issues upon delivery. We hope this project serves as a helpful resource for future engineers and wish you all great success and enjoyable development experiences ahead!

=== Project Video ===
Upload a video of your project and post the link here.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
We extend our heartfelt gratitude to Professor Preet and the ISA for organizing such an outstanding class and setting high standards. Their guidance and expectations continually motivated us to strive for excellence and surpass our own limits.

=== References Used ===
List any references used in project.

=== Appendix ===
You can list the references you used.

F24: Jardinains

2024-12-19T23:20:32Z

Proj user1: /* Acknowledgement */

=== Grading Criteria ===

* How well is Software & Hardware Design described?
* How well can this report be used to reproduce this project?
* Code Quality
* Overall Report Quality:
** Software Block Diagrams
** Hardware Block Diagrams
**: Schematic Quality
** Quality of technical challenges and solutions adopted.


== Project Title ==
Jardinaiins

== Abstract ==
Prepare for endless bouncing, breaking, and gnome-busting excitement! Jardinaiins reinvents classic brick-breaking gameplay with a delightful twist. Dive into vibrant, garden-themed levels filled with challenging obstacles, quirky gnome antics, and thrilling power-ups. Smash through barriers, outsmart mischievous gnomes, and uncover surprises as you progress in this fun and addictive adventure!

== Objectives & Introduction ==
• Interface the SJ2 board with the Waveshare 64x64 RGB LED Matrix Panel to display dynamic text, colorful images, and animations. The onboard GPIOs of the SJ2 microcontroller will drive the LED panel, and its chainable design will be utilized to facilitate scalability for larger displays.
• Use the MMA845XQ accelerometer onboard the SJ2 board as the player controller, mapping tilt and motion inputs to control animations, select media, or interact with displayed content on the LED matrix.
• Integrate the HiLetgo GD3300 UART Control Serial MP3 Music Player Module to enable digital audio output by leveraging the SJ2 board’s UART protocol. The system will send serial commands for operations like play, pause, track selection, and volume control.
• Utilize the General Purpose Direct Memory Access (GPDMA) controller on the SJ2 board for memory-to-memory transfers, reducing delays during video buffer updates for the LED matrix.
• Leverage the DMA controller's memory-to-peripheral function to minimize frequent I2S interrupts while loading digital audio content to the MP3 module, optimizing system performance.
• Implement an SD card peripheral driver on the SJ2 board to facilitate reading and writing of multimedia data, enabling seamless storage and retrieval of audio and visual content for playback and display.

=== Team Members & Responsibilities ===
* Dev Hitesh Malkan
**
* Nidhi Sharma
**
* Sunny Shah
**
* Navya
**

Sure! Here is the updated table with your provided data in the requested format:

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
* 10/14/2024
* 10/14/2024
|
* 10/20/2024
* 10/20/2024
|
* Review past projects, gather insights, and discuss with team members
* Initial ordering of parts (SJ2 board, 64x64 LED Matrix, GD3300 MP3 module).
|
* Completed
* Completed
|-
! scope="row"| 2
|
* 10/21/2024
* 10/21/2024
|
* 10/27/2024
* 10/27/2024
|
* Submit project proposals.
* Identify and select sensors and peripherals.
|
* Completed
* Completed
|-
! scope="row"| 3
|
* 10/28/2024
* 10/28/2024
|
* 11/03/2024
* 11/03/2024
|
* Learn about LED matrices and display "Jardinains."
* Study accelerometer principles and operation.
|
* Completed
* Completed
|-
! scope="row"| 4
|
* 11/04/2024
* 11/04/2024
* 11/04/2024
* 11/04/2024
|
* 11/10/2024
* 11/10/2024
* 11/10/2024
* 11/10/2024
|
* Create gitlab repo for the project.
* Develop basic game graphics and user interfaces on LED matrix.
* Develop code to control LED matrix based on accelerometer input.
* Familiarize with GD3300 MP3 decoder module.
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
* 11/11/2024
* 11/11/2024
* 11/11/2024
|
* 11/17/2024
* 11/17/2024
* 11/17/2024
|
* Design and implement game rules and systems.
* Integrate game logic with LED matrix display.
* Integrate sound effects with game logic.
|
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
* 11/18/2024
* 11/18/2024
* 11/18/2024
|
* 11/24/2024
* 11/24/2024
* 11/24/2024
|
* Test hardware integration and debug accelerometer-controlled input.
* Optimize DMA usage for efficient LED matrix updates.
* Validate UART-based communication with GD3300 MP3 module.
|
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/25/2024
* 11/25/2024
* 11/25/2024
|
* 12/01/2024
* 12/01/2024
* 12/01/2024
|
* Implement advanced game mechanics (e.g., power-ups, increasing difficulty).
* Optimize and fine-tune user experience and interactions.
* Test and validate system performance under real-time conditions.
|
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 12/02/2024
* 12/02/2024
* 12/02/2024
|
* 12/08/2024
* 12/08/2024
* 12/08/2024
|
* Finalize sprite designs and integrate animations with LED matrix.
* Conduct system-wide integration testing.
* Address edge cases in game logic and audio synchronization.
|
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 12/09/2024
* 12/09/2024
* 12/09/2024
|
* 12/17/2024
* 12/17/2024
* 12/17/2024
|
* Perform final testing of the complete system, ensuring stability and performance.
* Prepare and conduct final demonstrations.
* Submit final project code to GitLab and update project documentation and wiki.
|
* Completed
* Completed
* Completed
|-
|}

<HR>
 

== Parts List & Cost ==
{| class="wikitable"
|-
! scope="col"| #
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| SJ2 Board
| 1
| $50.00
|-
! scope="row"| 2
| Waveshare 64x64 4096 Pixels RGB Full-Color LED Matrix Panel
| Amazon
| 1
| $43.00
|-
! scope="row"| 3
| HiLetgo GD3300
| Amazon
| 1
| $8.00
|-
! scope="row"| 4
| SPST Normal Open ON/Off 2 Pin Round Button
| Amazon
| 3
| $8.00
|-
! scope="row"| 5
| MMA845XQ
| N/A
| 1
| N/A
|-
! scope="row"| 6
| SD card
| N/A
| 1
| N/A
|-
|}
<HR>
 

== Design & Implementation ==
The design section can go over your hardware and software design. Organize this section using sub-sections that go over your design and implementation.

=== Hardware Design ===
Discuss your hardware design here. Show detailed schematics, and the interface here.

=== Hardware Interface ===
In this section, you can describe how your hardware communicates, such as which BUSes used. You can discuss your driver implementation here, such that the '''Software Design''' section is isolated to talk about high level workings rather than inner working of your project.

=== Software Design ===
Controller
The accelerometer on the SJ2 board manages paddle movement by sampling the y-axis magnitude 17 times over a 17-millisecond period and calculating the average. This value is capped at a maximum of 700 within a range of 1000. To make it simpler, the tilt value is divided by 31, producing 32 possible readings, which correspond to 5 bits of data. Each time there is a change in magnitude or a button press, the controller transmits a single byte of data to the console. This byte encapsulates all the necessary information for controlling the on-screen paddle. The first two bits indicate button input, with 0 representing no input and 1 indicating input detection. The third bit specifies the tilt direction, where 0 means left and 1 means right. The remaining five bits represent the accelerometer's magnitude.

Audio
Sound effects are added using MP3 Serial Decoder HiLetgo GD3300, source code was written on our own.

=== Implementation ===
Paddle Movement
The paddle movement is driven by accelerometer data wirelessly transmitted from the SJ-Two board to the LPC4088 board. This data is mapped to 63 possible positions for the paddle. Initially, the paddle would instantly move to the position corresponding to the accelerometer input. However, this approach was refined by introducing an algorithm that allows for a smoother transition between positions. The algorithm calculates paddle velocity based on the "current desired position" and the previous position. Parameters of the algorithm were fine-tuned iteratively to achieve a smooth and responsive paddle movement.
Paddle-Ball Collision Logic
The ball's position is updated each frame by adding delta_x and delta_y to its current x and y coordinates. If the ball's next y-position crosses the paddle's y-position, a paddle collision check is performed. To do this, the ball's movement is modeled as a slope derived from delta_x and delta_y. Using this slope, along with the ball's current position and one pixel above the paddle's y-position, a point-slope equation (y2−y1)=m(x2−x1)(y_2 - y_1) = m(x_2 - x_1)(y2−y1)=m(x2−x1) is used to calculate the x-position of the ball at the time it reaches that y-coordinate.
Once the expected collision point is determined, the ball's collision magnitude and its next position's magnitude are calculated. By comparing these magnitudes, a time fraction for the frame is derived to estimate when the ball might have collided with the paddle. Using the paddle's current position and velocity, the paddle's previous position is estimated to determine how far it moved during this time fraction. By multiplying the time fraction by the paddle's x-position change, it is possible to check whether the paddle was underneath the ball at the moment it reached the pixel above the paddle's y-position.

== Testing & Technical Challenges ==
Ball-Collision Logic
The initial ball collision logic relied on pixel values to detect when the ball entered a brick. However, this approach failed to handle cases where the ball entered a brick diagonally, requiring the logic to be completely reworked. In the updated version, the focus shifted to determining whether the ball was crossing into an area containing a brick. This was achieved by dividing the playing field into a grid representing the positions of the bricks.
The new logic involved an initial check to see if the ball was crossing a boundary near a brick. If the ball crossed only the x-boundary, a collision check was performed to determine if it hit a brick. Similarly, if it crossed only the y-boundary, another collision check was conducted. In cases where the ball crossed both x- and y-boundaries, additional logic was applied to determine which boundary the ball crossed first. This updated method effectively handled diagonal collisions and improved the overall accuracy of the collision detection.
This experience highlighted the value of abandoning ineffective solutions and adopting new approaches, as they can lead to better outcomes.
Paddle Movement
We encountered an issue where the paddle would slow down as it neared the walls. This happened because the difference between the "current desired paddle position" and the previous paddle position decreased near the walls, resulting in a reduced paddle velocity. To address this, we adjusted the logic so that whenever the accelerometer data mapped to the minimum or maximum paddle index, the "current desired paddle position" was extended. This adjustment ensured that the paddle velocity remained consistent, preventing the paddle from slowing down as it approached the walls.
Paddle – Collision Logic
During final integration and testing, one major issue we faced was a bug where the ball failed to detect collisions with the paddle, causing it to pass through instead of bouncing off. To troubleshoot this, we implemented a clever debugging method: changing the ball's colour during each collision check. This allowed us to visually confirm whether the collision logic was being executed. Through this approach, we discovered that the section of code responsible for checking paddle collisions was being skipped, even when the ball's next y-position had crossed the paddle's y-position. This debugging method proved invaluable in identifying and resolving the root cause of the issue.

=== <Bug/issue name> ===
Discuss the issue and resolution.

== Conclusion ==
We thoroughly enjoyed the process of creating this project and gained a wealth of knowledge along the way. Our driver development skills were sharpened as we wrote drivers for various modules, including the LED matrix and MP3 decoder. Additionally, our understanding of the FreeRTOS API deepened, and we acquired fundamental game and graphics development skills through our work with the LED matrix.
Throughout the project, we faced numerous challenges, such as getting the LED matrix driver to function properly, implementing collision detection with multiple game objects, and designing and maintaining a complex game logic state machine, among many others. Each obstacle provided valuable learning experiences and contributed to our growth as developers. By applying design principles and communication protocols learned in lectures, we successfully brought this project to life. Key concepts we utilized include UART, SPI, I2C, I2S, DMA, FreeRTOS mutexes, binary semaphores, queues, task creation, stack sizing, task scheduling, interrupt service routines, and bootloaders.
Through countless hours of testing and verification, we learned the critical importance of allocating sufficient time for integration and debugging, as it greatly increases the likelihood of a successful outcome. Without this phase, the project might have faced significant issues upon delivery. We hope this project serves as a helpful resource for future engineers and wish you all great success and enjoyable development experiences ahead!

=== Project Video ===
Upload a video of your project and post the link here.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
We extend our heartfelt gratitude to Professor Preet and the ISA for organizing such an outstanding class and setting high standards. Their guidance and expectations continually motivated us to strive for excellence and surpass our own limits.

=== References Used ===
List any references used in project.

=== Appendix ===
You can list the references you used.

F24: Jardinains

2024-12-19T23:19:37Z

Proj user1: /* Parts List & Cost */

=== Grading Criteria ===

* How well is Software & Hardware Design described?
* How well can this report be used to reproduce this project?
* Code Quality
* Overall Report Quality:
** Software Block Diagrams
** Hardware Block Diagrams
**: Schematic Quality
** Quality of technical challenges and solutions adopted.


== Project Title ==
Jardinaiins

== Abstract ==
Prepare for endless bouncing, breaking, and gnome-busting excitement! Jardinaiins reinvents classic brick-breaking gameplay with a delightful twist. Dive into vibrant, garden-themed levels filled with challenging obstacles, quirky gnome antics, and thrilling power-ups. Smash through barriers, outsmart mischievous gnomes, and uncover surprises as you progress in this fun and addictive adventure!

== Objectives & Introduction ==
• Interface the SJ2 board with the Waveshare 64x64 RGB LED Matrix Panel to display dynamic text, colorful images, and animations. The onboard GPIOs of the SJ2 microcontroller will drive the LED panel, and its chainable design will be utilized to facilitate scalability for larger displays.
• Use the MMA845XQ accelerometer onboard the SJ2 board as the player controller, mapping tilt and motion inputs to control animations, select media, or interact with displayed content on the LED matrix.
• Integrate the HiLetgo GD3300 UART Control Serial MP3 Music Player Module to enable digital audio output by leveraging the SJ2 board’s UART protocol. The system will send serial commands for operations like play, pause, track selection, and volume control.
• Utilize the General Purpose Direct Memory Access (GPDMA) controller on the SJ2 board for memory-to-memory transfers, reducing delays during video buffer updates for the LED matrix.
• Leverage the DMA controller's memory-to-peripheral function to minimize frequent I2S interrupts while loading digital audio content to the MP3 module, optimizing system performance.
• Implement an SD card peripheral driver on the SJ2 board to facilitate reading and writing of multimedia data, enabling seamless storage and retrieval of audio and visual content for playback and display.

=== Team Members & Responsibilities ===
* Dev Hitesh Malkan
**
* Nidhi Sharma
**
* Sunny Shah
**
* Navya
**

Sure! Here is the updated table with your provided data in the requested format:

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
* 10/14/2024
* 10/14/2024
|
* 10/20/2024
* 10/20/2024
|
* Review past projects, gather insights, and discuss with team members
* Initial ordering of parts (SJ2 board, 64x64 LED Matrix, GD3300 MP3 module).
|
* Completed
* Completed
|-
! scope="row"| 2
|
* 10/21/2024
* 10/21/2024
|
* 10/27/2024
* 10/27/2024
|
* Submit project proposals.
* Identify and select sensors and peripherals.
|
* Completed
* Completed
|-
! scope="row"| 3
|
* 10/28/2024
* 10/28/2024
|
* 11/03/2024
* 11/03/2024
|
* Learn about LED matrices and display "Jardinains."
* Study accelerometer principles and operation.
|
* Completed
* Completed
|-
! scope="row"| 4
|
* 11/04/2024
* 11/04/2024
* 11/04/2024
* 11/04/2024
|
* 11/10/2024
* 11/10/2024
* 11/10/2024
* 11/10/2024
|
* Create gitlab repo for the project.
* Develop basic game graphics and user interfaces on LED matrix.
* Develop code to control LED matrix based on accelerometer input.
* Familiarize with GD3300 MP3 decoder module.
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
* 11/11/2024
* 11/11/2024
* 11/11/2024
|
* 11/17/2024
* 11/17/2024
* 11/17/2024
|
* Design and implement game rules and systems.
* Integrate game logic with LED matrix display.
* Integrate sound effects with game logic.
|
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
* 11/18/2024
* 11/18/2024
* 11/18/2024
|
* 11/24/2024
* 11/24/2024
* 11/24/2024
|
* Test hardware integration and debug accelerometer-controlled input.
* Optimize DMA usage for efficient LED matrix updates.
* Validate UART-based communication with GD3300 MP3 module.
|
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/25/2024
* 11/25/2024
* 11/25/2024
|
* 12/01/2024
* 12/01/2024
* 12/01/2024
|
* Implement advanced game mechanics (e.g., power-ups, increasing difficulty).
* Optimize and fine-tune user experience and interactions.
* Test and validate system performance under real-time conditions.
|
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 12/02/2024
* 12/02/2024
* 12/02/2024
|
* 12/08/2024
* 12/08/2024
* 12/08/2024
|
* Finalize sprite designs and integrate animations with LED matrix.
* Conduct system-wide integration testing.
* Address edge cases in game logic and audio synchronization.
|
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 12/09/2024
* 12/09/2024
* 12/09/2024
|
* 12/17/2024
* 12/17/2024
* 12/17/2024
|
* Perform final testing of the complete system, ensuring stability and performance.
* Prepare and conduct final demonstrations.
* Submit final project code to GitLab and update project documentation and wiki.
|
* Completed
* Completed
* Completed
|-
|}

<HR>
 

== Parts List & Cost ==
{| class="wikitable"
|-
! scope="col"| #
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| SJ2 Board
| 1
| $50.00
|-
! scope="row"| 2
| Waveshare 64x64 4096 Pixels RGB Full-Color LED Matrix Panel
| Amazon
| 1
| $43.00
|-
! scope="row"| 3
| HiLetgo GD3300
| Amazon
| 1
| $8.00
|-
! scope="row"| 4
| SPST Normal Open ON/Off 2 Pin Round Button
| Amazon
| 3
| $8.00
|-
! scope="row"| 5
| MMA845XQ
| N/A
| 1
| N/A
|-
! scope="row"| 6
| SD card
| N/A
| 1
| N/A
|-
|}
<HR>
 

== Design & Implementation ==
The design section can go over your hardware and software design. Organize this section using sub-sections that go over your design and implementation.

=== Hardware Design ===
Discuss your hardware design here. Show detailed schematics, and the interface here.

=== Hardware Interface ===
In this section, you can describe how your hardware communicates, such as which BUSes used. You can discuss your driver implementation here, such that the '''Software Design''' section is isolated to talk about high level workings rather than inner working of your project.

=== Software Design ===
Controller
The accelerometer on the SJ2 board manages paddle movement by sampling the y-axis magnitude 17 times over a 17-millisecond period and calculating the average. This value is capped at a maximum of 700 within a range of 1000. To make it simpler, the tilt value is divided by 31, producing 32 possible readings, which correspond to 5 bits of data. Each time there is a change in magnitude or a button press, the controller transmits a single byte of data to the console. This byte encapsulates all the necessary information for controlling the on-screen paddle. The first two bits indicate button input, with 0 representing no input and 1 indicating input detection. The third bit specifies the tilt direction, where 0 means left and 1 means right. The remaining five bits represent the accelerometer's magnitude.

Audio
Sound effects are added using MP3 Serial Decoder HiLetgo GD3300, source code was written on our own.

=== Implementation ===
Paddle Movement
The paddle movement is driven by accelerometer data wirelessly transmitted from the SJ-Two board to the LPC4088 board. This data is mapped to 63 possible positions for the paddle. Initially, the paddle would instantly move to the position corresponding to the accelerometer input. However, this approach was refined by introducing an algorithm that allows for a smoother transition between positions. The algorithm calculates paddle velocity based on the "current desired position" and the previous position. Parameters of the algorithm were fine-tuned iteratively to achieve a smooth and responsive paddle movement.
Paddle-Ball Collision Logic
The ball's position is updated each frame by adding delta_x and delta_y to its current x and y coordinates. If the ball's next y-position crosses the paddle's y-position, a paddle collision check is performed. To do this, the ball's movement is modeled as a slope derived from delta_x and delta_y. Using this slope, along with the ball's current position and one pixel above the paddle's y-position, a point-slope equation (y2−y1)=m(x2−x1)(y_2 - y_1) = m(x_2 - x_1)(y2−y1)=m(x2−x1) is used to calculate the x-position of the ball at the time it reaches that y-coordinate.
Once the expected collision point is determined, the ball's collision magnitude and its next position's magnitude are calculated. By comparing these magnitudes, a time fraction for the frame is derived to estimate when the ball might have collided with the paddle. Using the paddle's current position and velocity, the paddle's previous position is estimated to determine how far it moved during this time fraction. By multiplying the time fraction by the paddle's x-position change, it is possible to check whether the paddle was underneath the ball at the moment it reached the pixel above the paddle's y-position.

== Testing & Technical Challenges ==
Ball-Collision Logic
The initial ball collision logic relied on pixel values to detect when the ball entered a brick. However, this approach failed to handle cases where the ball entered a brick diagonally, requiring the logic to be completely reworked. In the updated version, the focus shifted to determining whether the ball was crossing into an area containing a brick. This was achieved by dividing the playing field into a grid representing the positions of the bricks.
The new logic involved an initial check to see if the ball was crossing a boundary near a brick. If the ball crossed only the x-boundary, a collision check was performed to determine if it hit a brick. Similarly, if it crossed only the y-boundary, another collision check was conducted. In cases where the ball crossed both x- and y-boundaries, additional logic was applied to determine which boundary the ball crossed first. This updated method effectively handled diagonal collisions and improved the overall accuracy of the collision detection.
This experience highlighted the value of abandoning ineffective solutions and adopting new approaches, as they can lead to better outcomes.
Paddle Movement
We encountered an issue where the paddle would slow down as it neared the walls. This happened because the difference between the "current desired paddle position" and the previous paddle position decreased near the walls, resulting in a reduced paddle velocity. To address this, we adjusted the logic so that whenever the accelerometer data mapped to the minimum or maximum paddle index, the "current desired paddle position" was extended. This adjustment ensured that the paddle velocity remained consistent, preventing the paddle from slowing down as it approached the walls.
Paddle – Collision Logic
During final integration and testing, one major issue we faced was a bug where the ball failed to detect collisions with the paddle, causing it to pass through instead of bouncing off. To troubleshoot this, we implemented a clever debugging method: changing the ball's colour during each collision check. This allowed us to visually confirm whether the collision logic was being executed. Through this approach, we discovered that the section of code responsible for checking paddle collisions was being skipped, even when the ball's next y-position had crossed the paddle's y-position. This debugging method proved invaluable in identifying and resolving the root cause of the issue.

=== <Bug/issue name> ===
Discuss the issue and resolution.

== Conclusion ==
We thoroughly enjoyed the process of creating this project and gained a wealth of knowledge along the way. Our driver development skills were sharpened as we wrote drivers for various modules, including the LED matrix and MP3 decoder. Additionally, our understanding of the FreeRTOS API deepened, and we acquired fundamental game and graphics development skills through our work with the LED matrix.
Throughout the project, we faced numerous challenges, such as getting the LED matrix driver to function properly, implementing collision detection with multiple game objects, and designing and maintaining a complex game logic state machine, among many others. Each obstacle provided valuable learning experiences and contributed to our growth as developers. By applying design principles and communication protocols learned in lectures, we successfully brought this project to life. Key concepts we utilized include UART, SPI, I2C, I2S, DMA, FreeRTOS mutexes, binary semaphores, queues, task creation, stack sizing, task scheduling, interrupt service routines, and bootloaders.
Through countless hours of testing and verification, we learned the critical importance of allocating sufficient time for integration and debugging, as it greatly increases the likelihood of a successful outcome. Without this phase, the project might have faced significant issues upon delivery. We hope this project serves as a helpful resource for future engineers and wish you all great success and enjoyable development experiences ahead!

=== Project Video ===
Upload a video of your project and post the link here.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Any acknowledgement that you may wish to provide can be included here.

=== References Used ===
List any references used in project.

=== Appendix ===
You can list the references you used.

F24: Jardinains

2024-12-19T23:13:31Z

Proj user1: /* Schedule */

=== Grading Criteria ===

* How well is Software & Hardware Design described?
* How well can this report be used to reproduce this project?
* Code Quality
* Overall Report Quality:
** Software Block Diagrams
** Hardware Block Diagrams
**: Schematic Quality
** Quality of technical challenges and solutions adopted.


== Project Title ==
Jardinaiins

== Abstract ==
Prepare for endless bouncing, breaking, and gnome-busting excitement! Jardinaiins reinvents classic brick-breaking gameplay with a delightful twist. Dive into vibrant, garden-themed levels filled with challenging obstacles, quirky gnome antics, and thrilling power-ups. Smash through barriers, outsmart mischievous gnomes, and uncover surprises as you progress in this fun and addictive adventure!

== Objectives & Introduction ==
• Interface the SJ2 board with the Waveshare 64x64 RGB LED Matrix Panel to display dynamic text, colorful images, and animations. The onboard GPIOs of the SJ2 microcontroller will drive the LED panel, and its chainable design will be utilized to facilitate scalability for larger displays.
• Use the MMA845XQ accelerometer onboard the SJ2 board as the player controller, mapping tilt and motion inputs to control animations, select media, or interact with displayed content on the LED matrix.
• Integrate the HiLetgo GD3300 UART Control Serial MP3 Music Player Module to enable digital audio output by leveraging the SJ2 board’s UART protocol. The system will send serial commands for operations like play, pause, track selection, and volume control.
• Utilize the General Purpose Direct Memory Access (GPDMA) controller on the SJ2 board for memory-to-memory transfers, reducing delays during video buffer updates for the LED matrix.
• Leverage the DMA controller's memory-to-peripheral function to minimize frequent I2S interrupts while loading digital audio content to the MP3 module, optimizing system performance.
• Implement an SD card peripheral driver on the SJ2 board to facilitate reading and writing of multimedia data, enabling seamless storage and retrieval of audio and visual content for playback and display.

=== Team Members & Responsibilities ===
* Dev Hitesh Malkan
**
* Nidhi Sharma
**
* Sunny Shah
**
* Navya
**

Sure! Here is the updated table with your provided data in the requested format:

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
* 10/14/2024
* 10/14/2024
|
* 10/20/2024
* 10/20/2024
|
* Review past projects, gather insights, and discuss with team members
* Initial ordering of parts (SJ2 board, 64x64 LED Matrix, GD3300 MP3 module).
|
* Completed
* Completed
|-
! scope="row"| 2
|
* 10/21/2024
* 10/21/2024
|
* 10/27/2024
* 10/27/2024
|
* Submit project proposals.
* Identify and select sensors and peripherals.
|
* Completed
* Completed
|-
! scope="row"| 3
|
* 10/28/2024
* 10/28/2024
|
* 11/03/2024
* 11/03/2024
|
* Learn about LED matrices and display "Jardinains."
* Study accelerometer principles and operation.
|
* Completed
* Completed
|-
! scope="row"| 4
|
* 11/04/2024
* 11/04/2024
* 11/04/2024
* 11/04/2024
|
* 11/10/2024
* 11/10/2024
* 11/10/2024
* 11/10/2024
|
* Create gitlab repo for the project.
* Develop basic game graphics and user interfaces on LED matrix.
* Develop code to control LED matrix based on accelerometer input.
* Familiarize with GD3300 MP3 decoder module.
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
* 11/11/2024
* 11/11/2024
* 11/11/2024
|
* 11/17/2024
* 11/17/2024
* 11/17/2024
|
* Design and implement game rules and systems.
* Integrate game logic with LED matrix display.
* Integrate sound effects with game logic.
|
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
* 11/18/2024
* 11/18/2024
* 11/18/2024
|
* 11/24/2024
* 11/24/2024
* 11/24/2024
|
* Test hardware integration and debug accelerometer-controlled input.
* Optimize DMA usage for efficient LED matrix updates.
* Validate UART-based communication with GD3300 MP3 module.
|
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/25/2024
* 11/25/2024
* 11/25/2024
|
* 12/01/2024
* 12/01/2024
* 12/01/2024
|
* Implement advanced game mechanics (e.g., power-ups, increasing difficulty).
* Optimize and fine-tune user experience and interactions.
* Test and validate system performance under real-time conditions.
|
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 12/02/2024
* 12/02/2024
* 12/02/2024
|
* 12/08/2024
* 12/08/2024
* 12/08/2024
|
* Finalize sprite designs and integrate animations with LED matrix.
* Conduct system-wide integration testing.
* Address edge cases in game logic and audio synchronization.
|
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 12/09/2024
* 12/09/2024
* 12/09/2024
|
* 12/17/2024
* 12/17/2024
* 12/17/2024
|
* Perform final testing of the complete system, ensuring stability and performance.
* Prepare and conduct final demonstrations.
* Submit final project code to GitLab and update project documentation and wiki.
|
* Completed
* Completed
* Completed
|-
|}

<HR>
 

== Parts List & Cost ==
Give a simple list of the cost of your project broken down by components. Do not write long stories here.

== Design & Implementation ==
The design section can go over your hardware and software design. Organize this section using sub-sections that go over your design and implementation.

=== Hardware Design ===
Discuss your hardware design here. Show detailed schematics, and the interface here.

=== Hardware Interface ===
In this section, you can describe how your hardware communicates, such as which BUSes used. You can discuss your driver implementation here, such that the '''Software Design''' section is isolated to talk about high level workings rather than inner working of your project.

=== Software Design ===
Controller
The accelerometer on the SJ2 board manages paddle movement by sampling the y-axis magnitude 17 times over a 17-millisecond period and calculating the average. This value is capped at a maximum of 700 within a range of 1000. To make it simpler, the tilt value is divided by 31, producing 32 possible readings, which correspond to 5 bits of data. Each time there is a change in magnitude or a button press, the controller transmits a single byte of data to the console. This byte encapsulates all the necessary information for controlling the on-screen paddle. The first two bits indicate button input, with 0 representing no input and 1 indicating input detection. The third bit specifies the tilt direction, where 0 means left and 1 means right. The remaining five bits represent the accelerometer's magnitude.

Audio
Sound effects are added using MP3 Serial Decoder HiLetgo GD3300, source code was written on our own.

=== Implementation ===
Paddle Movement
The paddle movement is driven by accelerometer data wirelessly transmitted from the SJ-Two board to the LPC4088 board. This data is mapped to 63 possible positions for the paddle. Initially, the paddle would instantly move to the position corresponding to the accelerometer input. However, this approach was refined by introducing an algorithm that allows for a smoother transition between positions. The algorithm calculates paddle velocity based on the "current desired position" and the previous position. Parameters of the algorithm were fine-tuned iteratively to achieve a smooth and responsive paddle movement.
Paddle-Ball Collision Logic
The ball's position is updated each frame by adding delta_x and delta_y to its current x and y coordinates. If the ball's next y-position crosses the paddle's y-position, a paddle collision check is performed. To do this, the ball's movement is modeled as a slope derived from delta_x and delta_y. Using this slope, along with the ball's current position and one pixel above the paddle's y-position, a point-slope equation (y2−y1)=m(x2−x1)(y_2 - y_1) = m(x_2 - x_1)(y2−y1)=m(x2−x1) is used to calculate the x-position of the ball at the time it reaches that y-coordinate.
Once the expected collision point is determined, the ball's collision magnitude and its next position's magnitude are calculated. By comparing these magnitudes, a time fraction for the frame is derived to estimate when the ball might have collided with the paddle. Using the paddle's current position and velocity, the paddle's previous position is estimated to determine how far it moved during this time fraction. By multiplying the time fraction by the paddle's x-position change, it is possible to check whether the paddle was underneath the ball at the moment it reached the pixel above the paddle's y-position.

== Testing & Technical Challenges ==
Ball-Collision Logic
The initial ball collision logic relied on pixel values to detect when the ball entered a brick. However, this approach failed to handle cases where the ball entered a brick diagonally, requiring the logic to be completely reworked. In the updated version, the focus shifted to determining whether the ball was crossing into an area containing a brick. This was achieved by dividing the playing field into a grid representing the positions of the bricks.
The new logic involved an initial check to see if the ball was crossing a boundary near a brick. If the ball crossed only the x-boundary, a collision check was performed to determine if it hit a brick. Similarly, if it crossed only the y-boundary, another collision check was conducted. In cases where the ball crossed both x- and y-boundaries, additional logic was applied to determine which boundary the ball crossed first. This updated method effectively handled diagonal collisions and improved the overall accuracy of the collision detection.
This experience highlighted the value of abandoning ineffective solutions and adopting new approaches, as they can lead to better outcomes.
Paddle Movement
We encountered an issue where the paddle would slow down as it neared the walls. This happened because the difference between the "current desired paddle position" and the previous paddle position decreased near the walls, resulting in a reduced paddle velocity. To address this, we adjusted the logic so that whenever the accelerometer data mapped to the minimum or maximum paddle index, the "current desired paddle position" was extended. This adjustment ensured that the paddle velocity remained consistent, preventing the paddle from slowing down as it approached the walls.
Paddle – Collision Logic
During final integration and testing, one major issue we faced was a bug where the ball failed to detect collisions with the paddle, causing it to pass through instead of bouncing off. To troubleshoot this, we implemented a clever debugging method: changing the ball's colour during each collision check. This allowed us to visually confirm whether the collision logic was being executed. Through this approach, we discovered that the section of code responsible for checking paddle collisions was being skipped, even when the ball's next y-position had crossed the paddle's y-position. This debugging method proved invaluable in identifying and resolving the root cause of the issue.

=== <Bug/issue name> ===
Discuss the issue and resolution.

== Conclusion ==
We thoroughly enjoyed the process of creating this project and gained a wealth of knowledge along the way. Our driver development skills were sharpened as we wrote drivers for various modules, including the LED matrix and MP3 decoder. Additionally, our understanding of the FreeRTOS API deepened, and we acquired fundamental game and graphics development skills through our work with the LED matrix.
Throughout the project, we faced numerous challenges, such as getting the LED matrix driver to function properly, implementing collision detection with multiple game objects, and designing and maintaining a complex game logic state machine, among many others. Each obstacle provided valuable learning experiences and contributed to our growth as developers. By applying design principles and communication protocols learned in lectures, we successfully brought this project to life. Key concepts we utilized include UART, SPI, I2C, I2S, DMA, FreeRTOS mutexes, binary semaphores, queues, task creation, stack sizing, task scheduling, interrupt service routines, and bootloaders.
Through countless hours of testing and verification, we learned the critical importance of allocating sufficient time for integration and debugging, as it greatly increases the likelihood of a successful outcome. Without this phase, the project might have faced significant issues upon delivery. We hope this project serves as a helpful resource for future engineers and wish you all great success and enjoyable development experiences ahead!

=== Project Video ===
Upload a video of your project and post the link here.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Any acknowledgement that you may wish to provide can be included here.

=== References Used ===
List any references used in project.

=== Appendix ===
You can list the references you used.

F24: Jardinains

2024-12-19T23:09:21Z

Proj user1:

=== Grading Criteria ===

* How well is Software & Hardware Design described?
* How well can this report be used to reproduce this project?
* Code Quality
* Overall Report Quality:
** Software Block Diagrams
** Hardware Block Diagrams
**: Schematic Quality
** Quality of technical challenges and solutions adopted.


== Project Title ==
Jardinaiins

== Abstract ==
Prepare for endless bouncing, breaking, and gnome-busting excitement! Jardinaiins reinvents classic brick-breaking gameplay with a delightful twist. Dive into vibrant, garden-themed levels filled with challenging obstacles, quirky gnome antics, and thrilling power-ups. Smash through barriers, outsmart mischievous gnomes, and uncover surprises as you progress in this fun and addictive adventure!

== Objectives & Introduction ==
• Interface the SJ2 board with the Waveshare 64x64 RGB LED Matrix Panel to display dynamic text, colorful images, and animations. The onboard GPIOs of the SJ2 microcontroller will drive the LED panel, and its chainable design will be utilized to facilitate scalability for larger displays.
• Use the MMA845XQ accelerometer onboard the SJ2 board as the player controller, mapping tilt and motion inputs to control animations, select media, or interact with displayed content on the LED matrix.
• Integrate the HiLetgo GD3300 UART Control Serial MP3 Music Player Module to enable digital audio output by leveraging the SJ2 board’s UART protocol. The system will send serial commands for operations like play, pause, track selection, and volume control.
• Utilize the General Purpose Direct Memory Access (GPDMA) controller on the SJ2 board for memory-to-memory transfers, reducing delays during video buffer updates for the LED matrix.
• Leverage the DMA controller's memory-to-peripheral function to minimize frequent I2S interrupts while loading digital audio content to the MP3 module, optimizing system performance.
• Implement an SD card peripheral driver on the SJ2 board to facilitate reading and writing of multimedia data, enabling seamless storage and retrieval of audio and visual content for playback and display.

=== Team Members & Responsibilities ===
* Dev Hitesh Malkan
**
* Nidhi Sharma
**
* Sunny Shah
**
* Navya
**

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
* 10/14/2024
|
* 10/20/2024
|
* Review past projects, identify key insights, and collaborate with team members.
|
* Completed
|-
! scope="row"| 2
|
*10/21/2024
*10/21/2024
|
*10/27/2024
*10/27/2024
|
*Identify and select necessary sensors and peripherals.
*Place orders for necessary parts.
|
* Completed
* Completed
|-
! scope="row"| 3
|
*10/28/2024
*10/28/2024
|
*11/03/2024
*11/03/2024
|
*Learn about LED matrices and display "Jardinains."
*Study accelerometer principles and operation.
|
* Completed
* completed
|-
! scope="row"| 4
|
* 11/04/2024
* 11/04/2024
* 11/04/2024
* 11/04/2024
|
* 11/10/2024
* 11/10/2024
* 11/10/2024
* 11/10/2024
|
* Create gitlab repo for the project.
* Create fundamental game graphics and user interfaces on LED matrix.
* Develop and implement code to control LED matrix based on accelerometer input.
* Get familiarized with MP3 decoder.
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
*11/11/2024
*11/11/2024
*11/11/2024
|
*11/17/2024
*11/17/2024
*11/17/2024
|
* Design and implement game rules and systems.
* Integrate game logic with LED matrix display.
* Integrate sound effects with game logic.
|
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
*11/18/2024
|
*11/24/2024
|
* Conduct comprehensive testing of code and hardware.
|
* completed
|-
! scope="row"| 7
|
* 11/25/2024
* 11/25/2024
|
* 12/01/2024
* 12/01/2024
|
* Commit and push final code to GitLab.
* Update and complete wiki page with project details.
|
* completed
* completed
|-
|}

<HR>
 

== Parts List & Cost ==
Give a simple list of the cost of your project broken down by components. Do not write long stories here.

== Design & Implementation ==
The design section can go over your hardware and software design. Organize this section using sub-sections that go over your design and implementation.

=== Hardware Design ===
Discuss your hardware design here. Show detailed schematics, and the interface here.

=== Hardware Interface ===
In this section, you can describe how your hardware communicates, such as which BUSes used. You can discuss your driver implementation here, such that the '''Software Design''' section is isolated to talk about high level workings rather than inner working of your project.

=== Software Design ===
Controller
The accelerometer on the SJ2 board manages paddle movement by sampling the y-axis magnitude 17 times over a 17-millisecond period and calculating the average. This value is capped at a maximum of 700 within a range of 1000. To make it simpler, the tilt value is divided by 31, producing 32 possible readings, which correspond to 5 bits of data. Each time there is a change in magnitude or a button press, the controller transmits a single byte of data to the console. This byte encapsulates all the necessary information for controlling the on-screen paddle. The first two bits indicate button input, with 0 representing no input and 1 indicating input detection. The third bit specifies the tilt direction, where 0 means left and 1 means right. The remaining five bits represent the accelerometer's magnitude.

Audio
Sound effects are added using MP3 Serial Decoder HiLetgo GD3300, source code was written on our own.

=== Implementation ===
Paddle Movement
The paddle movement is driven by accelerometer data wirelessly transmitted from the SJ-Two board to the LPC4088 board. This data is mapped to 63 possible positions for the paddle. Initially, the paddle would instantly move to the position corresponding to the accelerometer input. However, this approach was refined by introducing an algorithm that allows for a smoother transition between positions. The algorithm calculates paddle velocity based on the "current desired position" and the previous position. Parameters of the algorithm were fine-tuned iteratively to achieve a smooth and responsive paddle movement.
Paddle-Ball Collision Logic
The ball's position is updated each frame by adding delta_x and delta_y to its current x and y coordinates. If the ball's next y-position crosses the paddle's y-position, a paddle collision check is performed. To do this, the ball's movement is modeled as a slope derived from delta_x and delta_y. Using this slope, along with the ball's current position and one pixel above the paddle's y-position, a point-slope equation (y2−y1)=m(x2−x1)(y_2 - y_1) = m(x_2 - x_1)(y2−y1)=m(x2−x1) is used to calculate the x-position of the ball at the time it reaches that y-coordinate.
Once the expected collision point is determined, the ball's collision magnitude and its next position's magnitude are calculated. By comparing these magnitudes, a time fraction for the frame is derived to estimate when the ball might have collided with the paddle. Using the paddle's current position and velocity, the paddle's previous position is estimated to determine how far it moved during this time fraction. By multiplying the time fraction by the paddle's x-position change, it is possible to check whether the paddle was underneath the ball at the moment it reached the pixel above the paddle's y-position.

== Testing & Technical Challenges ==
Ball-Collision Logic
The initial ball collision logic relied on pixel values to detect when the ball entered a brick. However, this approach failed to handle cases where the ball entered a brick diagonally, requiring the logic to be completely reworked. In the updated version, the focus shifted to determining whether the ball was crossing into an area containing a brick. This was achieved by dividing the playing field into a grid representing the positions of the bricks.
The new logic involved an initial check to see if the ball was crossing a boundary near a brick. If the ball crossed only the x-boundary, a collision check was performed to determine if it hit a brick. Similarly, if it crossed only the y-boundary, another collision check was conducted. In cases where the ball crossed both x- and y-boundaries, additional logic was applied to determine which boundary the ball crossed first. This updated method effectively handled diagonal collisions and improved the overall accuracy of the collision detection.
This experience highlighted the value of abandoning ineffective solutions and adopting new approaches, as they can lead to better outcomes.
Paddle Movement
We encountered an issue where the paddle would slow down as it neared the walls. This happened because the difference between the "current desired paddle position" and the previous paddle position decreased near the walls, resulting in a reduced paddle velocity. To address this, we adjusted the logic so that whenever the accelerometer data mapped to the minimum or maximum paddle index, the "current desired paddle position" was extended. This adjustment ensured that the paddle velocity remained consistent, preventing the paddle from slowing down as it approached the walls.
Paddle – Collision Logic
During final integration and testing, one major issue we faced was a bug where the ball failed to detect collisions with the paddle, causing it to pass through instead of bouncing off. To troubleshoot this, we implemented a clever debugging method: changing the ball's colour during each collision check. This allowed us to visually confirm whether the collision logic was being executed. Through this approach, we discovered that the section of code responsible for checking paddle collisions was being skipped, even when the ball's next y-position had crossed the paddle's y-position. This debugging method proved invaluable in identifying and resolving the root cause of the issue.

=== <Bug/issue name> ===
Discuss the issue and resolution.

== Conclusion ==
We thoroughly enjoyed the process of creating this project and gained a wealth of knowledge along the way. Our driver development skills were sharpened as we wrote drivers for various modules, including the LED matrix and MP3 decoder. Additionally, our understanding of the FreeRTOS API deepened, and we acquired fundamental game and graphics development skills through our work with the LED matrix.
Throughout the project, we faced numerous challenges, such as getting the LED matrix driver to function properly, implementing collision detection with multiple game objects, and designing and maintaining a complex game logic state machine, among many others. Each obstacle provided valuable learning experiences and contributed to our growth as developers. By applying design principles and communication protocols learned in lectures, we successfully brought this project to life. Key concepts we utilized include UART, SPI, I2C, I2S, DMA, FreeRTOS mutexes, binary semaphores, queues, task creation, stack sizing, task scheduling, interrupt service routines, and bootloaders.
Through countless hours of testing and verification, we learned the critical importance of allocating sufficient time for integration and debugging, as it greatly increases the likelihood of a successful outcome. Without this phase, the project might have faced significant issues upon delivery. We hope this project serves as a helpful resource for future engineers and wish you all great success and enjoyable development experiences ahead!

=== Project Video ===
Upload a video of your project and post the link here.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Any acknowledgement that you may wish to provide can be included here.

=== References Used ===
List any references used in project.

=== Appendix ===
You can list the references you used.

F24: Jardinains

2024-12-19T23:07:56Z

Proj user1:

=== Grading Criteria ===

* How well is Software & Hardware Design described?
* How well can this report be used to reproduce this project?
* Code Quality
* Overall Report Quality:
** Software Block Diagrams
** Hardware Block Diagrams
**: Schematic Quality
** Quality of technical challenges and solutions adopted.


== Project Title ==
Jardinaiins

== Abstract ==
Prepare for endless bouncing, breaking, and gnome-busting excitement! Jardinaiins reinvents classic brick-breaking gameplay with a delightful twist. Dive into vibrant, garden-themed levels filled with challenging obstacles, quirky gnome antics, and thrilling power-ups. Smash through barriers, outsmart mischievous gnomes, and uncover surprises as you progress in this fun and addictive adventure!

== Objectives & Introduction ==
• Interface the SJ2 board with the Waveshare 64x64 RGB LED Matrix Panel to display dynamic text, colorful images, and animations. The onboard GPIOs of the SJ2 microcontroller will drive the LED panel, and its chainable design will be utilized to facilitate scalability for larger displays.
• Use the MMA845XQ accelerometer onboard the SJ2 board as the player controller, mapping tilt and motion inputs to control animations, select media, or interact with displayed content on the LED matrix.
• Integrate the HiLetgo GD3300 UART Control Serial MP3 Music Player Module to enable digital audio output by leveraging the SJ2 board’s UART protocol. The system will send serial commands for operations like play, pause, track selection, and volume control.
• Utilize the General Purpose Direct Memory Access (GPDMA) controller on the SJ2 board for memory-to-memory transfers, reducing delays during video buffer updates for the LED matrix.
• Leverage the DMA controller's memory-to-peripheral function to minimize frequent I2S interrupts while loading digital audio content to the MP3 module, optimizing system performance.
• Implement an SD card peripheral driver on the SJ2 board to facilitate reading and writing of multimedia data, enabling seamless storage and retrieval of audio and visual content for playback and display.

=== Team Members & Responsibilities ===
* Dev Hitesh Malkan
**
* Nidhi Sharma
**
* Sunny Shah
**
* Navya
**

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
* 10/14/2024
|
* 10/20/2024
|
* Review past projects, identify key insights, and collaborate with team members.
|
* Completed
|-
! scope="row"| 2
|
*10/21/2024
*10/21/2024
|
*10/27/2024
*10/27/2024
|
*Identify and select necessary sensors and peripherals.
*Place orders for necessary parts.
|
* Completed
* Completed
|-
! scope="row"| 3
|
*10/28/2024
*10/28/2024
|
*11/03/2024
*11/03/2024
|
*Learn about LED matrices and display "Jardinains."
*Study accelerometer principles and operation.
|
* Completed
* In progress
|-
! scope="row"| 4
|
* 11/04/2024
* 11/04/2024
* 11/04/2024
* 11/04/2024
|
* 11/10/2024
* 11/10/2024
* 11/10/2024
* 11/10/2024
|
* Create gitlab repo for the project.
* Create fundamental game graphics and user interfaces on LED matrix.
* Develop and implement code to control LED matrix based on accelerometer input.
* Get familiarized with MP3 decoder.
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
*11/11/2024
*11/11/2024
*11/11/2024
|
*11/17/2024
*11/17/2024
*11/17/2024
|
* Design and implement game rules and systems.
* Integrate game logic with LED matrix display.
* Integrate sound effects with game logic.
|
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
*11/18/2024
|
*11/24/2024
|
* Conduct comprehensive testing of code and hardware.
|
* Not started
|-
! scope="row"| 7
|
* 11/25/2024
* 11/25/2024
|
* 12/01/2024
* 12/01/2024
|
* Commit and push final code to GitLab.
* Update and complete wiki page with project details.
|
* completed
* completed
|-
|}

<HR>
 

== Parts List & Cost ==
Give a simple list of the cost of your project broken down by components. Do not write long stories here.

== Design & Implementation ==
The design section can go over your hardware and software design. Organize this section using sub-sections that go over your design and implementation.

=== Hardware Design ===
Discuss your hardware design here. Show detailed schematics, and the interface here.

=== Hardware Interface ===
In this section, you can describe how your hardware communicates, such as which BUSes used. You can discuss your driver implementation here, such that the '''Software Design''' section is isolated to talk about high level workings rather than inner working of your project.

=== Software Design ===
Controller
The accelerometer on the SJ2 board manages paddle movement by sampling the y-axis magnitude 17 times over a 17-millisecond period and calculating the average. This value is capped at a maximum of 700 within a range of 1000. To make it simpler, the tilt value is divided by 31, producing 32 possible readings, which correspond to 5 bits of data. Each time there is a change in magnitude or a button press, the controller transmits a single byte of data to the console. This byte encapsulates all the necessary information for controlling the on-screen paddle. The first two bits indicate button input, with 0 representing no input and 1 indicating input detection. The third bit specifies the tilt direction, where 0 means left and 1 means right. The remaining five bits represent the accelerometer's magnitude.

Audio
Sound effects are added using MP3 Serial Decoder HiLetgo GD3300, source code was written on our own.

=== Implementation ===
Paddle Movement
The paddle movement is driven by accelerometer data wirelessly transmitted from the SJ-Two board to the LPC4088 board. This data is mapped to 63 possible positions for the paddle. Initially, the paddle would instantly move to the position corresponding to the accelerometer input. However, this approach was refined by introducing an algorithm that allows for a smoother transition between positions. The algorithm calculates paddle velocity based on the "current desired position" and the previous position. Parameters of the algorithm were fine-tuned iteratively to achieve a smooth and responsive paddle movement.
Paddle-Ball Collision Logic
The ball's position is updated each frame by adding delta_x and delta_y to its current x and y coordinates. If the ball's next y-position crosses the paddle's y-position, a paddle collision check is performed. To do this, the ball's movement is modeled as a slope derived from delta_x and delta_y. Using this slope, along with the ball's current position and one pixel above the paddle's y-position, a point-slope equation (y2−y1)=m(x2−x1)(y_2 - y_1) = m(x_2 - x_1)(y2−y1)=m(x2−x1) is used to calculate the x-position of the ball at the time it reaches that y-coordinate.
Once the expected collision point is determined, the ball's collision magnitude and its next position's magnitude are calculated. By comparing these magnitudes, a time fraction for the frame is derived to estimate when the ball might have collided with the paddle. Using the paddle's current position and velocity, the paddle's previous position is estimated to determine how far it moved during this time fraction. By multiplying the time fraction by the paddle's x-position change, it is possible to check whether the paddle was underneath the ball at the moment it reached the pixel above the paddle's y-position.

== Testing & Technical Challenges ==
Ball-Collision Logic
The initial ball collision logic relied on pixel values to detect when the ball entered a brick. However, this approach failed to handle cases where the ball entered a brick diagonally, requiring the logic to be completely reworked. In the updated version, the focus shifted to determining whether the ball was crossing into an area containing a brick. This was achieved by dividing the playing field into a grid representing the positions of the bricks.
The new logic involved an initial check to see if the ball was crossing a boundary near a brick. If the ball crossed only the x-boundary, a collision check was performed to determine if it hit a brick. Similarly, if it crossed only the y-boundary, another collision check was conducted. In cases where the ball crossed both x- and y-boundaries, additional logic was applied to determine which boundary the ball crossed first. This updated method effectively handled diagonal collisions and improved the overall accuracy of the collision detection.
This experience highlighted the value of abandoning ineffective solutions and adopting new approaches, as they can lead to better outcomes.
Paddle Movement
We encountered an issue where the paddle would slow down as it neared the walls. This happened because the difference between the "current desired paddle position" and the previous paddle position decreased near the walls, resulting in a reduced paddle velocity. To address this, we adjusted the logic so that whenever the accelerometer data mapped to the minimum or maximum paddle index, the "current desired paddle position" was extended. This adjustment ensured that the paddle velocity remained consistent, preventing the paddle from slowing down as it approached the walls.
Paddle – Collision Logic
During final integration and testing, one major issue we faced was a bug where the ball failed to detect collisions with the paddle, causing it to pass through instead of bouncing off. To troubleshoot this, we implemented a clever debugging method: changing the ball's colour during each collision check. This allowed us to visually confirm whether the collision logic was being executed. Through this approach, we discovered that the section of code responsible for checking paddle collisions was being skipped, even when the ball's next y-position had crossed the paddle's y-position. This debugging method proved invaluable in identifying and resolving the root cause of the issue.

=== <Bug/issue name> ===
Discuss the issue and resolution.

== Conclusion ==
We thoroughly enjoyed the process of creating this project and gained a wealth of knowledge along the way. Our driver development skills were sharpened as we wrote drivers for various modules, including the LED matrix and MP3 decoder. Additionally, our understanding of the FreeRTOS API deepened, and we acquired fundamental game and graphics development skills through our work with the LED matrix.
Throughout the project, we faced numerous challenges, such as getting the LED matrix driver to function properly, implementing collision detection with multiple game objects, and designing and maintaining a complex game logic state machine, among many others. Each obstacle provided valuable learning experiences and contributed to our growth as developers. By applying design principles and communication protocols learned in lectures, we successfully brought this project to life. Key concepts we utilized include UART, SPI, I2C, I2S, DMA, FreeRTOS mutexes, binary semaphores, queues, task creation, stack sizing, task scheduling, interrupt service routines, and bootloaders.
Through countless hours of testing and verification, we learned the critical importance of allocating sufficient time for integration and debugging, as it greatly increases the likelihood of a successful outcome. Without this phase, the project might have faced significant issues upon delivery. We hope this project serves as a helpful resource for future engineers and wish you all great success and enjoyable development experiences ahead!

=== Project Video ===
Upload a video of your project and post the link here.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Any acknowledgement that you may wish to provide can be included here.

=== References Used ===
List any references used in project.

=== Appendix ===
You can list the references you used.

F24: Jardinains

2024-11-26T22:38:44Z

Proj user1: /* Schedule */

F24: Jardinains

2024-11-26T22:36:09Z

Proj user1: /* Abstract */

F24: Jardinains

2024-11-26T22:35:40Z

Proj user1: /* Project Title */

F24: Jardinains

2024-11-06T03:55:16Z

Proj user1:

F24: Jardinains

2024-11-01T22:26:27Z

Proj user1: Created page with "=== Grading Criteria === * How well is Software & Hardware Design described? * How well can this report be used to reproduce this project? * Code Quali..."

Realtime OS on Embedded Systems

2024-11-01T22:24:30Z

Proj user1: /* Fall 2024 */

== Program History ==
My contribution in Embedded System courses started with CmpE146. This course teaches students on how to write UART, SPI, and I2C Drivers and interface their drivers to peripherals. There are about 8 weekly labs in which students not only write drivers, but also learn the core of Real-time Operating Systems including threading (tasks), and inter-task communication using Queues and Semaphores. FreeRTOS is the operating system used with C/C++ Compiler based on GNU.

When the course was started by Dr. Ozemek @ SJSU (sometime before 2005), not many resources were out there. Still, with helpful guidance from Dr. Ozemek, interesting projects were created. This is when I started to help out in Embedded System Courses, and by collecting and sharing knowledge, we've raised the bar at SJSU!

There have been many great projects at SJSU, but since no one knew about them, the hard work went to a waste for anyone but the creator. But now we've got the infrastructure to share the projects, which turn out as great references for future students. Here is my project that started around 2007, and turned into Bachelor's Senior Design Project: 
[http://www.youtube.com/watch?v=QEadXdRl3ws&feature=plcp YouTube Video of Self-Navigating Car]

As of 2013, I have broadened my contribution to other embedded system courses like CmpE240, CmpE243 and CmpE244.

== Lab Assignments ==
This article contains laboratory assignments and resources. The assignments are under construction as we move towards SJ-One development board.
* [[Embedded System Tutorial Tasks | Lesson 0: Multiple Tasks]]
* [[Embedded System Tutorial GPIO | Lesson 1 : GPIO]]
* [[Embedded System Tutorial UART | Lesson 2 : UART]]
* [[Embedded System Tutorial SPI | Lesson 3 : SPI]]
* [[Embedded System I2C Tutorial | Lesson 4 : I2C]]
* [[Embedded System Tutorial Interrupts | Lesson 5 : Interrupts]]
* [[Embedded System Tutorial FreeRTOS | Lesson 6 : FreeRTOS Tasks]]
* [[Embedded System Tutorial File I/O | Lesson 7 : FreeRTOS Application Programming]]

==== Class Project ====
* [[MP3 Player]]

==Other reference articles==
* [[Bitmasking Tutorial]] (+ GPIO Example)
* [[ LPC17xx Memory Map & Interrupts]]

 
== Senior Design Projects ==

 
== Semester Projects ==
Every semester, students are given about 7-10 weeks to complete their projects. During this short time-span, students form groups, order parts, and begin working on their projects. The work performed during the semester is documented at this Wiki.

Here is the list of Preet's documented projects: 
* [[Preet's Relay Controller Project]]
* [[Nordic Low Powered Mesh Network stack]]
* [http://www.youtube.com/watch?v=QEadXdRl3ws&feature=plcp Senior Design Project (MS-CmpE) Video]

Here is another resource for good project references :
[http://people.ece.cornell.edu/land/courses/ece4760/FinalProjects/ Cornell EE476 Projects]

 
<HR>
=== Appendix ===

=== [[Fall 2024 | Fall 2024]] ===

* [[F24: Dodge Cars]]
* [[F24: Rival Rush]]
* [[F24: Space 6]]
* [[F24: Bounce Tales]]
* [[F24: Tilt Maze]]
* [[F24: Jardinains]]

=== [[Fall 2022 | Fall 2022]] ===

* [[F22: DDRTOS]]
* [[F22: xGameCreate]]
* [[F22: Embedded Gamers]]
* [[F22: Bob Burgers]]
* [[F22: Space Warriors]]
* [[F22: Thunder Flash]]

=== [[Fall 2021 | Fall 2021]] ===

* [[F21: ACE MARIO]]
* [[F21: Skeh-lleybones]]
* [[F21: Sons of Ultron]]
* [[F21: Juvenile Jumpers]]
* [[F21: Treasure Dive]]
* [[F21: FollowMe]]
* [[F21: Flame Over]]
* [[F21: Space Rage]]

=== [[Fall 2020 | Fall 2020]] ===
* [[F20: Jubeat]]
* [[F20: Flappy]]
* [[F20: Space Invaders]]
* [[F20: Treasure Diver]]
* [[F20: Tom & Jerry]]
* [[F20: Hungry Snake]]
* [[F20: Son of a Gun]]
* [[F20: Corona Run]]
* [[F20: Maddening Marbles]]
* [[F20: Bubble Shooter]]

=== [[Fall 2019 | Fall 2019]] ===

* [[F19: Smacman]]
* [[F19: Tetris]]
* [[F19: Space Impact]]
* [[F19: Road Max Fury]]
* [[F19: T-Rex Run!]]
* [[F19: Infinity Mirror]]
* [[F19: Tower Defense in Space]]
* [[F19: Alien Wars]]
* [[F19: Pocket tank]]
* [[F19: M&B (Morph & Blend)]]

=== [[Fall 2018 | Fall 2018]] ===

* [[F18: Flappy Bird]]
* [[F18: Catch me if you can]]
* [[F18: Zero Zero UFO]]
* [[F18: Goals of Glory]]
* [[F18: Spartan Warrior]]
* [[F18: Wireless sensor network]]
* [[F18: baem geim]]
* [[F18: Hit the Balloon]]
* [[F18: 2048]]

 

=== [[Spring 2018 | Spring 2018]] ===

* [[S18: Team Nemesis(SJSU Cam)]]
* [[S18: Death Race]]
* [[S18: Hand gesture controlled multiplayer game]]
* [[S18: Smart Rock Paper Scissors]]
* [[S18: Spybot]]
* [[S18: Audio Spectrum Analyzer with Graphics Display]]
* [[S18: XY-Plotter]]
* [[S18: RGB LED Sound Behavior on a Skateboard]]
* [[S18: Spark - MP3 Music Player with Audio Spectrum]]
* [[S18: Traffic Menace Video Game]]
* [[S18: M.E.O.W]]
* [[S18: Rahee]]

=== [[Spring 2017 | Spring 2017]] ===

* [[S17: Wake up Barista]]
* [[S17: Propeller Clock]]
* [[S17: Sky Knight]]
* [[S17: MyAutoHealth]]
* [[S17: Logan]]
* [[S17: ElectricBoard]]
* [[S17: CamBot]]
* [[S17: Sphero Droid]]
* [[S17: Smart Planter]]
* [[S17: Boom-Z Equalizer]]
* [[S17: Interactive Snake and ladder ]]
* [[S17: Smart Health Gear]]
* [[S17: Halo]]
* [[S17: Squad]]
* [[S17: Smart security system]]
 

=== [[Fall 2016 | Fall 2016]] ===

CMPE146:
* [http://www.socialledge.com/sjsu/index.php?title=F16:_Seismograph F16: Seismograph]
* [[F16: Piano Glove]]
* [[F16: Object Detector]]
* [[F16: Autonomous Nautical System]]
* [[F16: Autonomous Fire Extinguishing Vehicle]]
* [[F16: Autonomous Runaway Alarm Car]]
* [[F16: E-Bike]]
* [[F16: NotifyBox]]
* [[F16: Wireless Tilt Controlled Camera Arm]]
* [[F16: OBD2 Reader]]
* [[F16: Micro Watch Monitoring System]]
* [[F16: Door Alarm System]]
* [[http://www.socialledge.com/sjsu/index.php?title=F16:_I2Coffee F16: UART Coffee]]
* [[F16: SJone to FPGA wireless integration]]

=== [[Spring 2016 | Spring 2016]] ===
* [[S16: Fantastic Four]]
* [[S16: Simpsons]]
* [[S16: Mars 1]]
* [[S16: OpenSJ Bluz]]
* [[S16: Motion Copy Bot]]
* [[S16: Biker Assist]]
* [[S16: Helios]]
* [[S16: Sound Buddy]]
* [[S16: Warriors]]
* [[S16: Expendables]]
* [[S16: Ahava]]
* [[S16: Number 1]]
* [[S16: SkyNet]]
* [[S16: SmartDoorLock]]

Cmpe 146:
* [[S16: Camera Gimbal]]
* [[S16: Laser Harp]]
* <strike>[[S16: Laser Cutter Motor Controller]]</strike>
* [[S16: Sprinkler]]
* [[S16: The Jatrick Car]]
* [[S16: Dan]]
* [[S16: Robolamp]]
* [[S16: Pinball]]
 
<HR>

=== [[Fall 2015 | Fall 2015]] ===

CmpE146:
* [[F15: Autonomous Mobile]]
* [[F15: Car Report]]
* [[F15: Electronic Piano]]
* [[F15: Doorknock over Bluetooth]]
* [[F15: Smart Car]]
* [[F15: Plant Control]]
* [[F15: Laser Security System]]

 
<HR>

=== [[CmpE244 Spring 2015 | Spring 2015]] ===

 
* [[S15: Quadcopter - It flies]]
* [[S15: Remote Learner]]
* [[S15: Protocol Interface: I2C - CAN Bridge]]
* [[S15: Vision RC Car]]
* [[S15: SJeight Octocopter]]
* [[S15: Swarm Robots]]
* [[S15: Smart Sparta Parking System]]
* [[S15: Touch Navigator]]
* [[S15: Wizard's Chess System]]
* [[S15: Bug Rider]]
* [[S15: Real Time Brake Assist (RTBA)]]
* [[S15: Wireless Mesh Network]]
* [[S15: Wireless Power Transfer System]]
* [[S15: Drone]]
* [[S15: Tree Node using Google Protocol Buffers]]
* [[S15: Multi-media Car]]
* [[S15: Hand Gesture Recognition using IR Sensors]]
* [[S15: CAN controlled RGB LED cubes]]
* [[S15: Rubik's Cube Solver]]
* [[S15: RFID Security Box]]
* [[S15: Automated Meeting Room Reservation]]
* [[S15: Patient Buddy System (PBS)]]

CmpE146:
* [[S15: Hovercopter]]
* [[S15: Triclops: Smart RC Car]]
* [[S15: Connect Four - Robotic Player]]
* [[S15: Self-Balancing Robot]]
* [[S15: MP3 Player with Graphic Equalizer Display]]
* [[S15: Motion-Controlled RC Car]]
* [[S15: MENL (Monster Encounter Night Light) ]]
* [[S15: Tilt Motion Controlled LED Alarm Clock]]
* [[S15: Alarm Based Coffee Maker]]

=== [[CmpE244 Spring 2014 | Spring 2014]] ===

 
* Senior Project: [[Project Advising: Remote Security System]]

 
* [[S14: Quadcopter]]
* [[S14: Smart Weather Clock]]
* [[S14: Divine WINd]]
* [[S14: Data Acquisition using CAN bus]]
* [[S14: E-Ink Display for Shopping Tags]]
* [[S14: Spectrum Analyzer for Audio Frequency Signals]]
* [[S14: CAN Firmware Uploader]]
* [[S14: Asset Management and Location System]]
* [[S14: Location Tracker]]
* [[S14: Androbot]]
* [[S14: Virtual Dog]]
* [[S14: Android based Automation]]
* [[S14: FaceTime Robo]]
* [[S14: Wireless Control Car]]
* [[S14: Power Efficient Security Door System for Light-rail using CAN Bus]]
* [[S14: Android based home monitoring system]]
* [[S14: Need For Speed]]

CmpE146
* [[S14: Hyperintelligent NFC Locker of the Future]]
* [[S14: Smart Planter]]
* [[S14: Modular Security System]]
* [[S14: Autonomous Control System]]
* [[S14: Anti-Crash Car]]
* [[S14: Tricopter]]

=== [[CmpE240 Fall 2013 | Fall 2013]] ===

* [[F13: POV Display]]
* [[F13: Line Following Robot]]
* [[F13: LED Display]]
* [[F13: Bulb Ramper]]
* [[F13: Garage Parking Assistant]]
* [[F13: Quadcopter]]
* [[F13: BarkMaster2000]]
* [[F13: Remote Control Car]]
* [[F13: Obstacle Avoidance Robot]]
* [[F13: Vehicle On Board Diagnostics]]

=== [[CmpE146 Spring 2013 | Spring 2013]] ===

* [[S13: 2D Plotter]]
* [[S13: Smart Cube]]
* [[S13: Garage Parking Aid]]
* [[S13: Smart Security]]
* [[S13: Door Alarm System]]
* [[S13: Solar Panel Tracker]]

=== [[CmpE146 Fall 2012|Fall 2012]] ===

* [[F12: Evil Watchdog]]
* [[F12: Smart Bulb]]
* [[F12: All Your Base are Belong to You]]
* [[F12: Android Controlled MP3]]
* [[F12: Unified Wireless Health Monitoring System]]
* [[F12: OBD-II Android Monitor]]
* [[F12: Self-Driving GPS Following Car]]
* [[F12: Android Door Lock]]

=== [[CmpE146 Spring 2012|Spring 2012]] ===
* [[S12: FreeRTOS based QuadCopter]]
* [[S12: Web-based MP3 Player]]
* [[S12: Self Drive Car]]
* [[S12: VAndroid]]
* [[S12: Traffic Light Sensing Vehicle]]
* [[S12: Sound Reader]]
* [[S12: Remote Controlled MP3 Player]]
* [[S12: Android Controlled Robot]]
* [[S12: Eyes-Free GPS]]

 

== Handy References ==
* [[Sample Project Report]]
* [[Project Proposal Guidelines]]
* [[CmpE146 Lab. Resources]]

F21: Treasure Dive

2021-12-18T13:18:39Z

Proj user1: /* Conclusion */

== '''Treasure Dive''' ==
[[File:mermaid.png|thumb|500px|right|Game Screen Capture]]
[[File:Cmpe-244-controller-pic.png|thumb|600px|right|Game Controller]]

== Abstract ==
Treasure Dive is a modern one player game inspired by the popular arcade game Breakout, which was released by Atari in 1976. The ancients rambled about an unconfirmed tale of a shipwreck that saw no survivors. As an adventurous and skilled free diver, you are seeking the lost treasure buried deep in an ocean abyss. You dove down all the way to the seafloor and found the buried treasure but need to make it back to the surface to see another day! The player uses a wireless controller to control an on-screen paddle to break bricks on the top of the screen. When enough bricks are broken, a passage is revealed, allowing you to get closer to the surface.

== Objectives & Introduction ==
The objective of this project is to:
 
- interface an LPC4088 to a VGA monitor by leveraging the LCD controller and to use an accelerometer as the sensor for the player controller
 
- integrate two Zigbees RF modules integrated for wireless communication between the controller and LPC4088 board via UART protocol
 
- enable digital audio output by developing an I2S driver and porting an audio tracker library
 
- utilize the General Purpose Direct Memory Access (GPDMA) controller memory-to-memory function to decrease memory write times when updating the video buffer
 
- utilize the DMA controller's memory-to-peripheral function to help reduce frequent I2S interrupts when loading digital audio contents
 
- implement an SD card peripheral driver to facilitate memory exchange between an SD card

=== Team Members & Responsibilities ===
[[File:Cmpe-244-brian-pic.png ‎]]
* '''Brian Ho'''
** Wiki project schedule planning
** Interfacing the accelerometer
** Digital to Analog Conversion resistor ladder design used to interface digital LCD controller signal with analog VGA port signal
** PCB planning and acquisition
** Procurement of controller hardware
** Wireless communication via Zigbees RF modules and UART protocol
** Wireless controller logic and integration
** Ball movement and wall/brick collision logic
** Hardware/software development and integration
** Final testing and integration
[[File:Cmpe-244-billy-pic.png]]
* '''[https://www.linkedin.com/in/william-lai-525681139 Billy Lai]'''
** Wiki project schedule planning
** Increased CPU clock rate from 96 MHz to 120 MHz
** Initial memory write timing and measurements to estimate memory write capacity
** DMA driver memory-to-memory implementation for transferring graphics data to video buffer
** I2S with DMA memory-to-peripheral implementation to reduce memory-write interrupts when I2S Tx FIFO level is low
** Paddle movement and paddle collision logic, testing and integration
** Software development and integration
** Final testing and integration
[[File:Cmpe-244-jasdip-pic.png]]
* '''Jasdip Sekhon'''
** Wiki project schedule planning
** PCB design and planning
** File loading method algorithm for storing loaded files in memory
** Audio track and audio effects selection for gameplay audio
** Palette manipulation for gameplay
** Software development and integration
** Final testing and integration
[[File:Cmpe-244-isaac-pic.png ]]
* '''Isaac Wahhab'''
** Wiki project schedule planning
** DAC resistor ladder design and integration
** Hardware procurement (LPC4088 board, Zigbees, VGA monitor, etc)
** VGA driving through LCD controller
** Audio tracker library port
** Digital audio output using I2S
** Palette scrolling feature implementation
** Graphics rendering
** Vertical scrolling implementation for the video buffer
** Sprite testing and implementation for the ball
** SD card peripheral driver for reading assets from SD card
** Art assets acquisition
** Hardware/software development and integration
** Gameplay logic design
** Gameplay audio and level design
** Final testing and integration

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
*09/21/2021
|
*09/27/2021
|
* Read previous projects, gather information and discuss among the group members.
* Initial ordering of parts (LPC 4088 board, 64x64 LED screen)
|
* Completed
* Completed
|-
! scope="row"| 2
|
*09/28/2021
|
*10/04/2021
|
* Submit project proposals
|
* Completed
|-
! scope="row"| 3
|
*10/05/2021
|
*10/11/2021
|
* Test if board can drive VGA
* Test varying clock rates
* Hardware cursor on VGA
* Rough display driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 4
|
* 10/12/2021
|
* 10/18/2021
|
* DAC resistor ladder
* RGB channels
* Palette shifting
* Measure memory buffer writing timing
* Wiki schedule planning
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
*10/19/2021
|
*10/25/2021
|
* Calculate upper-bound for VGA refresh timings
* Create GitLab repository for project
* Joystick functionality
* SD card driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
*10/26/2021
|
*11/01/2021
|
* Audio peripheral
* Wireless communication using Zigbee module
* PCB planning
* Accelerometer driver
* Continue SD card driver
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/02/2021

|
* 11/08/2021
|
* Continue audio peripheral
* Continue wireless communication using Zigbee module
* Continue accelerometer driver
* PCB finalizing and ordering
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 11/09/2021
|
* 11/15/2021
|
* Convert PNG data to graphics format
* Game background functionality
* Sprite functionality
* User interface
* Integrate components
* Initial testing
|
* Completed
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 11/16/2021
|
* 11/22/2021
|
* Continue everything from last week
* Game physics (wall collision and AI movement)
* Measurements for wooden case
* Art assets (audio and visual)
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 10
|
*11/23/2021
|
*11/29/2021
|

* Fine-tune user experience
* Make case
|
* Completed
* Completed
|-
! scope="row"| 11
|
* 11/30/2021
|
* 12/06/2021
|
* Continue fine-tune user experience
* Integrate hardware
|
* Completed
* Completed
|-
! scope="row"| 12
|
* 12/07/2021
|
* 12/16/2021
|
* Final testing
* Final demo
|
* Completed
* Completed
|-
|}

== Bill Of Materials ==
{| class="wikitable" style="margin-left: 0px; margin-right: auto;"
|-
! scope="col"| Item#
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| [https://www.embeddedartists.com/products/lpc4088-developers-kit LPC4088-32 Developer's Kit]
| 1
| $300.00
|-
! scope="row"| 2
| Microcontroller Board
| [http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems/page/sj2-board SJ2 Board]
| 1
| $50.00
|-
! scope="row"| 3
| Bluetooth Module
| [https://hub.digi.com/support/products/digi-xbee/digi-xbee-s1 Digi XBee-S1]
| 2
| N/A
|-
! scope="row"| 4
| DAC PCB
| [https://jlcpcb.com JLCPCB (Set of 5)]
| 1
| $15.00
|-
! scope="row"| 4
| Monitor
| Asus TFT Monitor
| 1
| N/A
|-
! scope="row"| 5
| Mini Breadboard
| [https://www.amazon.com/EL-CK-002-Electronic-Breadboard-Capacitor-Potentiometer/dp/B01ERP6WL4/ref=pd_sbs_147_2/132-6936347-2117569?_encoding=UTF8&pd_rd_i=B01ERP6WL4&pd_rd_r=6a783095-ec87-42ae-9f01-a5ebc9a44607&pd_rd_w=V2aU6&pd_rd_wg=11h1i&pf_rd_p=d66372fe-68a6-48a3-90ec-41d7f64212be&pf_rd_r=368WNKXFBBQY2FJQ4119&psc=1&refRID=368WNKXFBBQY2FJQ4119 ELEGOO Mini Breadboard]
| 1
| $13.99
|-
! scope="row"| 6
| Arcade buttons
| [https://www.amazon.com/EG-STARTS-Illuminated-Buttons-Raspberry/dp/B01N11BDX9?pd_rd_w=DiSpt&pf_rd_p=5bc65336-a8b0-4888-9e70-4d4ce2f6bd59&pf_rd_r=JCH2VQYXFPHND4XZ4FAM&pd_rd_r=8d6243dd-e1bd-4dae-9b19-be2e00f3085f&pd_rd_wg=K4lt9&pd_rd_i=B01N11BDX9&psc=1&ref_=pd_bap_d_rp_1_i EG STARTS LED Arcade Buttons]
| 2
| $11.99
|-
! scope="row"| 7
| VGA cable
| N/A
| 1
| N/A
|-
! scope="row"| 7
| SD card
| N/A
| 1
| N/A
|-
|}

== Design & Implementation ==

=== Hardware Design ===
We designed our PCB using Autodesk EAGLE based on the tutorial on PCB design during the class lecture. We ordered our PCB to be manufactured from JLCPCB. The PCB consists of an R-2R resistor ladder DAC for the purpose of converting the digital signal from the microcontroller to analog to display graphics on the VGA monitor. Since the VGA input for each color channel is 0-0.7V, the resistor ladder scaled the LPC4088 outputs into proper DAC values for the VGA to take in. Five bits are passed to each of the red, green, and blue channels of the VGA in addition to other signals, such as Horizontal sync (HSYNC) and Vertical sync (VSYNC).
 
[[File:pcbSchematic.png|500px|thumb|left|PCB Schematic]]
[[File:pcbLayout.png|500px|thumb|right|PCB Layout]]
[[File:pcbPic-min.png|500px|thumb|right|Manufactured PCB]]
[[File:pcbManufactured.png|500px|thumb|left|Soldered PCB]]
 

=== Hardware Interface ===

Two Digi XBee devices were used to communicate between the SJ2-Board controller and the LPC4088 board to control the paddle on the game. The XBee devices were programmed using the XTCU programming application provided by Digi International. The SJ2-Board used one XBee as a transmitted and the LPC4088 used another as a receiver. The two modules communicated using UART frames and a single byte represented any change of movement or button presses sent by the controller.

{| class="wikitable"
|-
! scope="col"| XCTU
! scope="col"| Transmitter
! scope="col"| Receiver
|-
! scope="row"| CH
| C
| C
|-
! scope="row"| ID
| 6969
| 6969
|-
! scope="row"| CE
| Coordinator
| Endpoint
|-
! scope="row"| Baud Rate
| 9600
| 9600
|-
|}
[[File:sj2Board.png|380px|thumb|center|SJ2 Board]]
[[File:CMPE146_F16_MW_XbeeModule.jpg|200px|thumb|center|XBee Device]]
 

Two LED buttons were used to interface with the controller as inputs. They were both set up using GPIO.
[[File:VGA LED Button.jpg|200px|thumb|center|LED Button]]
 
[[File:VGA_Controller_Block_Diagram.PNG|380px|thumb|center|Controller Block Diagram]]
 

==== I2S Peripheral and Stereo Audio Decoder ====

The LPC4088 is equipped with an I2S stereo encoder. This module provides a hardware-enabled solution for sending I2S-encoded audio data packets to stereo decoders, which produce the corresponding audio waveform. For this project, since 16-bit stereo audio playback was chosen, data packets transmitted on this interface were sent in the following format:

[[File:I2S_audio_data_word.png]]

Here, each word of data encodes a single packet of audio in two 16-bit sections: one signal level for the left channel, and another for the right. These 4 bytes are sent at a rate of 44100 Hz, resulting in a total data transmission of 176400 bytes a second. Rather than needing to be software-defined, this transmission is automatically handled by hardware using a 8-word deep FIFO queue that manages streaming out the audio data at a fixed rate, saving a significant amount of CPU cycles.

This encoded audio signal was sent to a dedicated external [https://www.nxp.com/docs/en/data-sheet/UDA1380.pdf stereo decoder], which output to a 3.5 mm audio jack connected to speakers in the VGA monitor.

==== LCD Peripheral ====

The LCD peripheral was configured for 16-bit color palette (5 bits of color data for each color level -- red, green, and blue -- and an unused intensity bit), at a resolution of 800 by 600 pixels, and with an active color palette size of 256 colors. Since the logic for driving an LCD display has the same concepts (horizontal blanking, vertical blanking, etc) and timing information for any given resolution was the same, we were able to directly drive a VGA display with the same signals and timing data (after the digital signal had been converted to analog).

[[File:LCD_%2B_VGA_Video_signal.png]]

The timing register configurations needed to achieve the requested video display was derived from a dedicated website for [http://tinyvga.com/vga-timing/800x600@60Hz VGA signal info].

==== External Memory ====

The game's visual assets and tracker audio file (both after being loaded from an SD card) and the game's video buffer required more memory than what is present natively in the microcontroller. To accommodate this, an external 32MiB memory module was used. This module was interfaced with using the external memory controller peripheral present on the board.

=== Software Design ===
==== Controller ====

There is an internal accelerometer on the SJ2 board that is utilized as the movement controls for the paddle movement. The y-axis value of the magnitude is polled 17 times in 17 milliseconds and is divided by 17 to get an average reading per that time frame. The maximum tilt value that can be read by the controller is set to 700 out of the total 1000 (highest possible value). The current tilt value is divided by 31 to get a total of 32 possible readings to represent 5 bits of magnitude.

The controller sends one byte of data to the console whenever there is any change in magnitude on the accelerometer or button changes. The byte contains all the information needed to control the paddle on the screen. The first two bits on the byte represent 0 if there is no button input and 1 if there is a button input. The third bit represents whether the controller is tilted left or right with 0 being left and 1 being right. The last five bits represent the controller accelerometer's magnitude.

[[File:VGA_UART_Frame.PNG]]

==== Audio ====

Sound design was provided to the project (in the form of music and sound effects) through the use of a [https://https://en.wikipedia.org/wiki/Music_tracker tracker audio library]. Source code was adopted from the [https://github.com/schismtracker/schismtracker Schism Tracker] library and altered with the following modifications:

- All UI logic was stripped from the source code

- All file handling logic was reworked to use the SD card reading functions, as the project does not support POSIX file calls

- The [https://www.libsdl.org/ Simple DirectMedia Layer] hooks were replaced with a FreeRTOS task that periodically called the waveform generation callback function, which was passed to the I2S peripheral hooked up to a stereo decoder.

=== Implementation ===

==== Ball Logic ====

The ball is controlled by the buttons and the controller accelerometer values. When the button is pressed, the ball is initially sent with a random velocity. The velocity is determined by a substate timer and is initially set as one of 6 possible values.

{| class="wikitable"
|-
! scope="col"| Substate Count
! scope="col"| X-Velocity
! scope="col"| Y-Velocity
|-
! scope="row"| 0
| 4
| -2
|-
! scope="row"| 1
| -4
| -2
|-
! scope="row"| 2
| 2
| -4
|-
! scope="row"| 3
| -2
| -4
|-
! scope="row"| 4
| 1
| -5
|-
! scope="row"| 5
| -1
| -5
|-
|}

The ball's velocity changes whenever it detects a collision with either the side of the screen, the bricks on the top of the level, or the paddle at the bottom. If colliding with the paddle, the ball's velocity will take into account the paddle's velocity and add it to its current velocity. This is done by adding the paddle's x-velocity and adjusting the y-velocity by normalizing the velocity vectors to have a magnitude of 5. This means the ball will always be moving with a magnitude of 5 in any direction after it hits the paddle. If the ball collides with anything other than the paddle, then the velocity of the ball is changed with no increase or decrease of magnitude. If the ball collides on the side of a wall or block, the x-velocity will reflect (go from negative to positive or vice-versa) and the y-velocity will remain constant. If the ball collides on the top or bottom of a block, the y-velocity will reflect and the x-velocity will remain constant. If it appears that the ball collides with the side and top or bottom of something, then the ball will check whichever face it will collide with first and then calculate the other face. Lastly, when the ball collides with the bottom of the screen, then the player experiences a "death" and the ball will reset on the paddle. If the ball collides with the top of the screen, the screen will scroll up for the user to play on the next level.

Whenever the ball collides with a brick, the brick will disappear. To accomplish this, there is a 2-D array of unsigned integers that contains the values of the bricks. If the integer is 0, then there is no block. If the integer is any value other than 0, then the block is filled and has collision logic. If the ball approaches a region that might have a block, then there will be a check within that 2-D to see if the block is filled or empty. If the block is filled, there will be the collision logic that changes the vector of the ball and sets the integer value in the 2-D array to 0 to represent that the block has been broken.

==== Paddle Movement ====
The paddle movement is controlled by the accelerometer data being wirelessly transmitted from the SJ-Two board to the LPC4088 board. The accelerometer data will map to 63 possible indexes for the paddle location. Originally, the paddle would jump directly into the corresponding index location depending on the accelerometer input. We later iterated on this process by implementing an algorithm to smoothly transition into these indexes. This was done by calculating a paddle velocity using the "current desired paddle position" and the previous paddle position. We adjusted the parameters of the algorithm iteratively to produce the smooth paddle movement.

==== Paddle Collision Logic ====
The position of the ball is calculated by adding delta_x and delta_y to the ball's current x and y position. For each frame, if the ball's next Y position crosses the paddle's Y position, we would check for a paddle collision. To do this, we used the delta_x and delta_y to get the ball's movement as a slope. We then substituted the slope, the ball's current position, and 1 pixel above the paddle's Y position into a point-slope equation (y2 - y1) = m(x2 - x1) to get the paddle's X position at the time of the ball reaching the pixel above paddle's Y position. After finding the location where the ball is expected to collide with the paddle, we then calculated the ball's collision magnitude and the ball's next location's magnitude. Using a ratio of these two magnitudes, we have a frame time fraction of when the ball possibly collided with the paddle. Then, by using the paddle's current position and paddle velocity, we are able to find where the paddle previously was, and measure how far the paddle had moved during that time frame. By multiplying the time frame from the ball's collision magnitude to the paddle's change in X position, we are able to find whether the paddle was under the ball at the time of the ball reaching the pixel above the paddle's Y position.

== Testing & Technical Challenges ==

=== VGA Interfacing ===
A DAC resistor ladder was designed and a corresponding custom PCB was acquired to interface the 3.3V digital LCD controller signal with the analog VGA input signal. The VGA cable takes between 0V (completely dark) and 0.7V (maximum brightness) for color pin inputs. The DAC resistor ladder was designed and implemented on a breadboard to verify the design. Validating the design on a breadboard gave us a clear vision during PCB design and planning.

=== VGA Write Timings ===
There were initial tests that were required to be done to see if using a VGA monitor as an output was a viable option for this project. We had to test memory write timings to see if they were fast enough to match the refresh rate of the monitor. Those tests would decide how much of the screen buffer we could write to during the blanking periods during the VGA refresh. This was a good learning experience for testing hardware capabilities and should be done whenever introducing any new aspects to a project.

=== Ball Collision Logic ===
The initial version of the ball collision used pixel values to see if the ball was entering the brick. The problem was that this logic did not account for the case where the ball enters the brick diagonally and the logic for that version of the collision had to be scrapped. The later iteration of the logic was changed to checking whether the ball was potentially crossing into an area that had a brick. This was done by separating the checks into a grid that represented the locations of the blocks. In this case, there was an initial check to see if the ball was crossing a boundary where there might be a new brick. If the ball is crossing the x-boundary and not the y-boundary, then there would be a collision check to see if it collided with a block. If it is crossing the y-boundary and not the x-boundary, once again then there would be a collision check to see if it collided with a block. If it is crossing both the x-boundary and y-boundary, then there would be a check to see which boundary the ball would cross first. This new logic allowed for us to account for the case where the block enters the block diagonally. This taught us that sometimes it is beneficial to scrap a previous iteration and use a new approach to solving a problem because it may lead to better results.

=== Paddle Movement ===
One issue we discovered was the paddle would slow down as it approached the walls. This was due to the difference in "current desired paddle position" and the previous paddle position becoming smaller as the walls were approached, leading to a smaller paddle velocity. To account for this, whenever the accelerometer sent data that mapped to the minimum or maximum paddle index, we would increase the distance of the "current desired paddle position" so the paddle velocity would not slow the paddle down as it approached the walls.

=== Paddle Collision Logic ===
During final integration and testing, one of the biggest issues we encountered was a bug where the ball would not detect a collision with the paddle. The ball would phase right through the paddle instead of bouncing off of it. One of the great ideas that we thought of was to change the color of the ball during each check to see if the checks would go through. This debug check allowed us to verify that section of code that checked for paddle collisions was being skipped even though the ball's next Y position had crossed the paddle's Y position. By having this debugging method, we were able to root cause the issue.

== Conclusion ==
For some of us, this was our first experience in designing and implementing a functional embedded system using FreeRTOS. The challenges we hurdled to accomplish our technical goals helped to shape a journey that was greatly enhanced by the opportunity to meet in person and having a hands-on approach to hardware and component integration. We were able to utilize design principles and implement communication protocols we learned in lecture to pull this project together. Some useful topics include: UART, SPI, I2C, I2S, DMA, FreeRTOS mutexes and binary semaphores, queues, task creation, stack size, task scheduler, interrupt service routines, bootloaders, etc. After extensive hours spent testing and verifying our implementation, we have learned that time allocated for integration and testing will significantly improve chances of a successful project. If we had skipped out on testing and verification, the project could have possibly crashed and burned when it was delivered. We hope this project can be of use to any future engineers following behind us, and we wish you all good times and happy development ahead!

=== Project Video ===
[https://www.youtube.com/watch?v=ipngAGlN86c SJSU CMPE 244 FALL 2021: Embedded Software Project Gameplay]

=== Project Source Code ===
* [https://gitlab.com/fayek.wahhab/sjtwo-c/-/tree/feature/i2s_dma2 Treasure Dive Baseline for Demo Source Code]
* [https://gitlab.com/hobriandh/sjtwo-c/-/tree/final_controller Controller for Demo Source Code]

=== Project Presentation ===
* [https://docs.google.com/presentation/d/1pIAAq51RySaZrniwJGuuHpVVPT1xaE4mjnMTCl6fnXA/edit?usp=sharing Treasure Dive Demo Presentation]

== References ==
=== Acknowledgement ===
We would like to acknowledge and thank our instructor Preetpal Kang for sharing his knowledge in embedded software development using FreeRTOS.
 
Another shoutout to our art designer Jacob for the unique artwork displayed in our game.

=== Appendix ===
*[https://gitlab.com/sjtwo-c-dev/sjtwo-c/-/blob/master/LPC408X_7X%20User%20Manual.pdf LPC408x/407x Datasheet]
*[https://www.freertos.org/Documentation/RTOS_book.html FreeRTOS Documentation]
*[http://elm-chan.org/fsw/ff/00index_e.html FatFs - Generic FAT Filesystem Module]
*[https://www.digi.com/resources/library/data-sheets/ds_xbeemultipointmodules Xbee Datasheet]
*[http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems Embedded Drivers & Real Time Operating Systems Lessons]

F21: Treasure Dive

2021-12-18T12:56:45Z

Proj user1: /* Abstract */

== '''Treasure Dive''' ==
[[File:mermaid.png|thumb|500px|right|Game Screen Capture]]
[[File:Cmpe-244-controller-pic.png|thumb|600px|right|Game Controller]]

== Abstract ==
Treasure Dive is a modern one player game inspired by the popular arcade game Breakout, which was released by Atari in 1976. The ancients rambled about an unconfirmed tale of a shipwreck that saw no survivors. As an adventurous and skilled free diver, you are seeking the lost treasure buried deep in an ocean abyss. You dove down all the way to the seafloor and found the buried treasure but need to make it back to the surface to see another day! The player uses a wireless controller to control an on-screen paddle to break bricks on the top of the screen. When enough bricks are broken, a passage is revealed, allowing you to get closer to the surface.

== Objectives & Introduction ==
The objective of this project is to:
 
- interface an LPC4088 to a VGA monitor by leveraging the LCD controller and to use an accelerometer as the sensor for the player controller
 
- integrate two Zigbees RF modules integrated for wireless communication between the controller and LPC4088 board via UART protocol
 
- enable digital audio output by developing an I2S driver and porting an audio tracker library
 
- utilize the General Purpose Direct Memory Access (GPDMA) controller memory-to-memory function to decrease memory write times when updating the video buffer
 
- utilize the DMA controller's memory-to-peripheral function to help reduce frequent I2S interrupts when loading digital audio contents
 
- implement an SD card peripheral driver to facilitate memory exchange between an SD card

=== Team Members & Responsibilities ===
[[File:Cmpe-244-brian-pic.png ‎]]
* '''Brian Ho'''
** Wiki project schedule planning
** Interfacing the accelerometer
** Digital to Analog Conversion resistor ladder design used to interface digital LCD controller signal with analog VGA port signal
** PCB planning and acquisition
** Procurement of controller hardware
** Wireless communication via Zigbees RF modules and UART protocol
** Wireless controller logic and integration
** Ball movement and wall/brick collision logic
** Hardware/software development and integration
** Final testing and integration
[[File:Cmpe-244-billy-pic.png]]
* '''[https://www.linkedin.com/in/william-lai-525681139 Billy Lai]'''
** Wiki project schedule planning
** Increased CPU clock rate from 96 MHz to 120 MHz
** Initial memory write timing and measurements to estimate memory write capacity
** DMA driver memory-to-memory implementation for transferring graphics data to video buffer
** I2S with DMA memory-to-peripheral implementation to reduce memory-write interrupts when I2S Tx FIFO level is low
** Paddle movement and paddle collision logic, testing and integration
** Software development and integration
** Final testing and integration
[[File:Cmpe-244-jasdip-pic.png]]
* '''Jasdip Sekhon'''
** Wiki project schedule planning
** PCB design and planning
** File loading method algorithm for storing loaded files in memory
** Audio track and audio effects selection for gameplay audio
** Palette manipulation for gameplay
** Software development and integration
** Final testing and integration
[[File:Cmpe-244-isaac-pic.png ]]
* '''Isaac Wahhab'''
** Wiki project schedule planning
** DAC resistor ladder design and integration
** Hardware procurement (LPC4088 board, Zigbees, VGA monitor, etc)
** VGA driving through LCD controller
** Audio tracker library port
** Digital audio output using I2S
** Palette scrolling feature implementation
** Graphics rendering
** Vertical scrolling implementation for the video buffer
** Sprite testing and implementation for the ball
** SD card peripheral driver for reading assets from SD card
** Art assets acquisition
** Hardware/software development and integration
** Gameplay logic design
** Gameplay audio and level design
** Final testing and integration

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
*09/21/2021
|
*09/27/2021
|
* Read previous projects, gather information and discuss among the group members.
* Initial ordering of parts (LPC 4088 board, 64x64 LED screen)
|
* Completed
* Completed
|-
! scope="row"| 2
|
*09/28/2021
|
*10/04/2021
|
* Submit project proposals
|
* Completed
|-
! scope="row"| 3
|
*10/05/2021
|
*10/11/2021
|
* Test if board can drive VGA
* Test varying clock rates
* Hardware cursor on VGA
* Rough display driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 4
|
* 10/12/2021
|
* 10/18/2021
|
* DAC resistor ladder
* RGB channels
* Palette shifting
* Measure memory buffer writing timing
* Wiki schedule planning
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
*10/19/2021
|
*10/25/2021
|
* Calculate upper-bound for VGA refresh timings
* Create GitLab repository for project
* Joystick functionality
* SD card driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
*10/26/2021
|
*11/01/2021
|
* Audio peripheral
* Wireless communication using Zigbee module
* PCB planning
* Accelerometer driver
* Continue SD card driver
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/02/2021

|
* 11/08/2021
|
* Continue audio peripheral
* Continue wireless communication using Zigbee module
* Continue accelerometer driver
* PCB finalizing and ordering
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 11/09/2021
|
* 11/15/2021
|
* Convert PNG data to graphics format
* Game background functionality
* Sprite functionality
* User interface
* Integrate components
* Initial testing
|
* Completed
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 11/16/2021
|
* 11/22/2021
|
* Continue everything from last week
* Game physics (wall collision and AI movement)
* Measurements for wooden case
* Art assets (audio and visual)
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 10
|
*11/23/2021
|
*11/29/2021
|

* Fine-tune user experience
* Make case
|
* Completed
* Completed
|-
! scope="row"| 11
|
* 11/30/2021
|
* 12/06/2021
|
* Continue fine-tune user experience
* Integrate hardware
|
* Completed
* Completed
|-
! scope="row"| 12
|
* 12/07/2021
|
* 12/16/2021
|
* Final testing
* Final demo
|
* Completed
* Completed
|-
|}

== Bill Of Materials ==
{| class="wikitable" style="margin-left: 0px; margin-right: auto;"
|-
! scope="col"| Item#
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| [https://www.embeddedartists.com/products/lpc4088-developers-kit LPC4088-32 Developer's Kit]
| 1
| $300.00
|-
! scope="row"| 2
| Microcontroller Board
| [http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems/page/sj2-board SJ2 Board]
| 1
| $50.00
|-
! scope="row"| 3
| Bluetooth Module
| [https://hub.digi.com/support/products/digi-xbee/digi-xbee-s1 Digi XBee-S1]
| 2
| N/A
|-
! scope="row"| 4
| DAC PCB
| [https://jlcpcb.com JLCPCB (Set of 5)]
| 1
| $15.00
|-
! scope="row"| 4
| Monitor
| Asus TFT Monitor
| 1
| N/A
|-
! scope="row"| 5
| Mini Breadboard
| [https://www.amazon.com/EL-CK-002-Electronic-Breadboard-Capacitor-Potentiometer/dp/B01ERP6WL4/ref=pd_sbs_147_2/132-6936347-2117569?_encoding=UTF8&pd_rd_i=B01ERP6WL4&pd_rd_r=6a783095-ec87-42ae-9f01-a5ebc9a44607&pd_rd_w=V2aU6&pd_rd_wg=11h1i&pf_rd_p=d66372fe-68a6-48a3-90ec-41d7f64212be&pf_rd_r=368WNKXFBBQY2FJQ4119&psc=1&refRID=368WNKXFBBQY2FJQ4119 ELEGOO Mini Breadboard]
| 1
| $13.99
|-
! scope="row"| 6
| Arcade buttons
| [https://www.amazon.com/EG-STARTS-Illuminated-Buttons-Raspberry/dp/B01N11BDX9?pd_rd_w=DiSpt&pf_rd_p=5bc65336-a8b0-4888-9e70-4d4ce2f6bd59&pf_rd_r=JCH2VQYXFPHND4XZ4FAM&pd_rd_r=8d6243dd-e1bd-4dae-9b19-be2e00f3085f&pd_rd_wg=K4lt9&pd_rd_i=B01N11BDX9&psc=1&ref_=pd_bap_d_rp_1_i EG STARTS LED Arcade Buttons]
| 2
| $11.99
|-
! scope="row"| 7
| VGA cable
| N/A
| 1
| N/A
|-
! scope="row"| 7
| SD card
| N/A
| 1
| N/A
|-
|}

== Design & Implementation ==

=== Hardware Design ===
We designed our PCB using Autodesk EAGLE based on the tutorial on PCB design during the class lecture. We ordered our PCB to be manufactured from JLCPCB. The PCB consists of an R-2R resistor ladder DAC for the purpose of converting the digital signal from the microcontroller to analog to display graphics on the VGA monitor. Since the VGA input for each color channel is 0-0.7V, the resistor ladder scaled the LPC4088 outputs into proper DAC values for the VGA to take in. Five bits are passed to each of the red, green, and blue channels of the VGA in addition to other signals, such as Horizontal sync (HSYNC) and Vertical sync (VSYNC).
 
[[File:pcbSchematic.png|500px|thumb|left|PCB Schematic]]
[[File:pcbLayout.png|500px|thumb|right|PCB Layout]]
[[File:pcbPic-min.png|500px|thumb|right|Manufactured PCB]]
[[File:pcbManufactured.png|500px|thumb|left|Soldered PCB]]
 

=== Hardware Interface ===

Two Digi XBee devices were used to communicate between the SJ2-Board controller and the LPC4088 board to control the paddle on the game. The XBee devices were programmed using the XTCU programming application provided by Digi International. The SJ2-Board used one XBee as a transmitted and the LPC4088 used another as a receiver. The two modules communicated using UART frames and a single byte represented any change of movement or button presses sent by the controller.

{| class="wikitable"
|-
! scope="col"| XCTU
! scope="col"| Transmitter
! scope="col"| Receiver
|-
! scope="row"| CH
| C
| C
|-
! scope="row"| ID
| 6969
| 6969
|-
! scope="row"| CE
| Coordinator
| Endpoint
|-
! scope="row"| Baud Rate
| 9600
| 9600
|-
|}
[[File:sj2Board.png|380px|thumb|center|SJ2 Board]]
[[File:CMPE146_F16_MW_XbeeModule.jpg|200px|thumb|center|XBee Device]]
 

Two LED buttons were used to interface with the controller as inputs. They were both set up using GPIO.
[[File:VGA LED Button.jpg|200px|thumb|center|LED Button]]
 
[[File:VGA_Controller_Block_Diagram.PNG|380px|thumb|center|Controller Block Diagram]]
 

==== I2S Peripheral and Stereo Audio Decoder ====

The LPC4088 is equipped with an I2S stereo encoder. This module provides a hardware-enabled solution for sending I2S-encoded audio data packets to stereo decoders, which produce the corresponding audio waveform. For this project, since 16-bit stereo audio playback was chosen, data packets transmitted on this interface were sent in the following format:

[[File:I2S_audio_data_word.png]]

Here, each word of data encodes a single packet of audio in two 16-bit sections: one signal level for the left channel, and another for the right. These 4 bytes are sent at a rate of 44100 Hz, resulting in a total data transmission of 176400 bytes a second. Rather than needing to be software-defined, this transmission is automatically handled by hardware using a 8-word deep FIFO queue that manages streaming out the audio data at a fixed rate, saving a significant amount of CPU cycles.

This encoded audio signal was sent to a dedicated external [https://www.nxp.com/docs/en/data-sheet/UDA1380.pdf stereo decoder], which output to a 3.5 mm audio jack connected to speakers in the VGA monitor.

==== LCD Peripheral ====

The LCD peripheral was configured for 16-bit color palette (5 bits of color data for each color level -- red, green, and blue -- and an unused intensity bit), at a resolution of 800 by 600 pixels, and with an active color palette size of 256 colors. Since the logic for driving an LCD display has the same concepts (horizontal blanking, vertical blanking, etc) and timing information for any given resolution was the same, we were able to directly drive a VGA display with the same signals and timing data (after the digital signal had been converted to analog).

[[File:LCD_%2B_VGA_Video_signal.png]]

The timing register configurations needed to achieve the requested video display was derived from a dedicated website for [http://tinyvga.com/vga-timing/800x600@60Hz VGA signal info].

==== External Memory ====

The game's visual assets and tracker audio file (both after being loaded from an SD card) and the game's video buffer required more memory than what is present natively in the microcontroller. To accommodate this, an external 32MiB memory module was used. This module was interfaced with using the external memory controller peripheral present on the board.

=== Software Design ===
==== Controller ====

There is an internal accelerometer on the SJ2 board that is utilized as the movement controls for the paddle movement. The y-axis value of the magnitude is polled 17 times in 17 milliseconds and is divided by 17 to get an average reading per that time frame. The maximum tilt value that can be read by the controller is set to 700 out of the total 1000 (highest possible value). The current tilt value is divided by 31 to get a total of 32 possible readings to represent 5 bits of magnitude.

The controller sends one byte of data to the console whenever there is any change in magnitude on the accelerometer or button changes. The byte contains all the information needed to control the paddle on the screen. The first two bits on the byte represent 0 if there is no button input and 1 if there is a button input. The third bit represents whether the controller is tilted left or right with 0 being left and 1 being right. The last five bits represent the controller accelerometer's magnitude.

[[File:VGA_UART_Frame.PNG]]

==== Audio ====

Sound design was provided to the project (in the form of music and sound effects) through the use of a [https://https://en.wikipedia.org/wiki/Music_tracker tracker audio library]. Source code was adopted from the [https://github.com/schismtracker/schismtracker Schism Tracker] library and altered with the following modifications:

- All UI logic was stripped from the source code

- All file handling logic was reworked to use the SD card reading functions, as the project does not support POSIX file calls

- The [https://www.libsdl.org/ Simple DirectMedia Layer] hooks were replaced with a FreeRTOS task that periodically called the waveform generation callback function, which was passed to the I2S peripheral hooked up to a stereo decoder.

=== Implementation ===

==== Ball Logic ====

The ball is controlled by the buttons and the controller accelerometer values. When the button is pressed, the ball is initially sent with a random velocity. The velocity is determined by a substate timer and is initially set as one of 6 possible values.

{| class="wikitable"
|-
! scope="col"| Substate Count
! scope="col"| X-Velocity
! scope="col"| Y-Velocity
|-
! scope="row"| 0
| 4
| -2
|-
! scope="row"| 1
| -4
| -2
|-
! scope="row"| 2
| 2
| -4
|-
! scope="row"| 3
| -2
| -4
|-
! scope="row"| 4
| 1
| -5
|-
! scope="row"| 5
| -1
| -5
|-
|}

The ball's velocity changes whenever it detects a collision with either the side of the screen, the bricks on the top of the level, or the paddle at the bottom. If colliding with the paddle, the ball's velocity will take into account the paddle's velocity and add it to its current velocity. This is done by adding the paddle's x-velocity and adjusting the y-velocity by normalizing the velocity vectors to have a magnitude of 5. This means the ball will always be moving with a magnitude of 5 in any direction after it hits the paddle. If the ball collides with anything other than the paddle, then the velocity of the ball is changed with no increase or decrease of magnitude. If the ball collides on the side of a wall or block, the x-velocity will reflect (go from negative to positive or vice-versa) and the y-velocity will remain constant. If the ball collides on the top or bottom of a block, the y-velocity will reflect and the x-velocity will remain constant. If it appears that the ball collides with the side and top or bottom of something, then the ball will check whichever face it will collide with first and then calculate the other face. Lastly, when the ball collides with the bottom of the screen, then the player experiences a "death" and the ball will reset on the paddle. If the ball collides with the top of the screen, the screen will scroll up for the user to play on the next level.

Whenever the ball collides with a brick, the brick will disappear. To accomplish this, there is a 2-D array of unsigned integers that contains the values of the bricks. If the integer is 0, then there is no block. If the integer is any value other than 0, then the block is filled and has collision logic. If the ball approaches a region that might have a block, then there will be a check within that 2-D to see if the block is filled or empty. If the block is filled, there will be the collision logic that changes the vector of the ball and sets the integer value in the 2-D array to 0 to represent that the block has been broken.

==== Paddle Movement ====
The paddle movement is controlled by the accelerometer data being wirelessly transmitted from the SJ-Two board to the LPC4088 board. The accelerometer data will map to 63 possible indexes for the paddle location. Originally, the paddle would jump directly into the corresponding index location depending on the accelerometer input. We later iterated on this process by implementing an algorithm to smoothly transition into these indexes. This was done by calculating a paddle velocity using the "current desired paddle position" and the previous paddle position. We adjusted the parameters of the algorithm iteratively to produce the smooth paddle movement.

==== Paddle Collision Logic ====
The position of the ball is calculated by adding delta_x and delta_y to the ball's current x and y position. For each frame, if the ball's next Y position crosses the paddle's Y position, we would check for a paddle collision. To do this, we used the delta_x and delta_y to get the ball's movement as a slope. We then substituted the slope, the ball's current position, and 1 pixel above the paddle's Y position into a point-slope equation (y2 - y1) = m(x2 - x1) to get the paddle's X position at the time of the ball reaching the pixel above paddle's Y position. After finding the location where the ball is expected to collide with the paddle, we then calculated the ball's collision magnitude and the ball's next location's magnitude. Using a ratio of these two magnitudes, we have a frame time fraction of when the ball possibly collided with the paddle. Then, by using the paddle's current position and paddle velocity, we are able to find where the paddle previously was, and measure how far the paddle had moved during that time frame. By multiplying the time frame from the ball's collision magnitude to the paddle's change in X position, we are able to find whether the paddle was under the ball at the time of the ball reaching the pixel above the paddle's Y position.

== Testing & Technical Challenges ==

=== VGA Interfacing ===
A DAC resistor ladder was designed and a corresponding custom PCB was acquired to interface the 3.3V digital LCD controller signal with the analog VGA input signal. The VGA cable takes between 0V (completely dark) and 0.7V (maximum brightness) for color pin inputs. The DAC resistor ladder was designed and implemented on a breadboard to verify the design. Validating the design on a breadboard gave us a clear vision during PCB design and planning.

=== VGA Write Timings ===
There were initial tests that were required to be done to see if using a VGA monitor as an output was a viable option for this project. We had to test memory write timings to see if they were fast enough to match the refresh rate of the monitor. Those tests would decide how much of the screen buffer we could write to during the blanking periods during the VGA refresh. This was a good learning experience for testing hardware capabilities and should be done whenever introducing any new aspects to a project.

=== Ball Collision Logic ===
The initial version of the ball collision used pixel values to see if the ball was entering the brick. The problem was that this logic did not account for the case where the ball enters the brick diagonally and the logic for that version of the collision had to be scrapped. The later iteration of the logic was changed to checking whether the ball was potentially crossing into an area that had a brick. This was done by separating the checks into a grid that represented the locations of the blocks. In this case, there was an initial check to see if the ball was crossing a boundary where there might be a new brick. If the ball is crossing the x-boundary and not the y-boundary, then there would be a collision check to see if it collided with a block. If it is crossing the y-boundary and not the x-boundary, once again then there would be a collision check to see if it collided with a block. If it is crossing both the x-boundary and y-boundary, then there would be a check to see which boundary the ball would cross first. This new logic allowed for us to account for the case where the block enters the block diagonally. This taught us that sometimes it is beneficial to scrap a previous iteration and use a new approach to solving a problem because it may lead to better results.

=== Paddle Movement ===
One issue we discovered was the paddle would slow down as it approached the walls. This was due to the difference in "current desired paddle position" and the previous paddle position becoming smaller as the walls were approached, leading to a smaller paddle velocity. To account for this, whenever the accelerometer sent data that mapped to the minimum or maximum paddle index, we would increase the distance of the "current desired paddle position" so the paddle velocity would not slow the paddle down as it approached the walls.

=== Paddle Collision Logic ===
During final integration and testing, one of the biggest issues we encountered was a bug where the ball would not detect a collision with the paddle. The ball would phase right through the paddle instead of bouncing off of it. One of the great ideas that we thought of was to change the color of the ball during each check to see if the checks would go through. This debug check allowed us to verify that section of code that checked for paddle collisions was being skipped even though the ball's next Y position had crossed the paddle's Y position. By having this debugging method, we were able to root cause the issue.

== Conclusion ==
For some of us, this was our first experience in designing and implementing a functional embedded system using FreeRTOS. The challenges we hurdled to accomplish our technical goals helped to shape a journey that was greatly enhanced by the opportunity to meet in person and having a hands-on approach to hardware and component integration. We were able to utilize design principles and implement communication protocols we learned in lecture to pull this project together. Some useful topics include: UART, SPI, I2C, I2S, DMA, FreeRTOS mutexes and binary semaphores, queues, task creation, stack size, task scheduler, interrupt service routines, bootloaders, etc. After extensive hours spent testing and verifying our implementation, we have learned that time allocated for integration and testing will significantly improve chances of a successful project. If we had skipped out on testing and verification, the project could have possibly crashed and burned when it was delivered. We hope this project can be of use to any future engineers following behind us, and we wish you all good times and happy development ahead!

=== Project Video ===
[https://www.youtube.com/watch?v=ipngAGlN86c SJSU CMPE 244 FALL 2021: Embedded Software Project Gameplay]

=== Project Source Code ===
* [https://gitlab.com/fayek.wahhab/sjtwo-c/-/tree/feature/i2s_dma2 Treasure Dive Baseline for Demo Source Code]
* [https://gitlab.com/hobriandh/sjtwo-c/-/tree/final_controller Controller for Demo Source Code]

== References ==
=== Acknowledgement ===
We would like to acknowledge and thank our instructor Preetpal Kang for sharing his knowledge in embedded software development using FreeRTOS.
 
Another shoutout to our art designer Jacob for the unique artwork displayed in our game.

=== Appendix ===
*[https://gitlab.com/sjtwo-c-dev/sjtwo-c/-/blob/master/LPC408X_7X%20User%20Manual.pdf LPC408x/407x Datasheet]
*[https://www.freertos.org/Documentation/RTOS_book.html FreeRTOS Documentation]
*[http://elm-chan.org/fsw/ff/00index_e.html FatFs - Generic FAT Filesystem Module]
*[https://www.digi.com/resources/library/data-sheets/ds_xbeemultipointmodules Xbee Datasheet]
*[http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems Embedded Drivers & Real Time Operating Systems Lessons]

F21: Treasure Dive

2021-12-18T12:54:16Z

Proj user1: /* Team Members & Responsibilities */

== '''Treasure Dive''' ==
[[File:mermaid.png|thumb|500px|right|Game Screen Capture]]
[[File:Cmpe-244-controller-pic.png|thumb|600px|right|Game Controller]]

== Abstract ==
Treasure Dive is a modern one player game inspired by the popular arcade game Breakout, which was released by Atari in 1976. The ancients rambled about an unconfirmed tale of a shipwreck that saw no survivors. As an adventurous and skilled free diver, you are seeking treasure buried deep in an ocean abyss. You dove down all the way to the seafloor and found the buried treasure but need to make it back to the surface to see another day! The player uses a wireless controller to control an on-screen paddle to break bricks on the top of the screen. When enough bricks are broken, a passage is revealed, allowing you to get closer to the surface.

== Objectives & Introduction ==
The objective of this project is to:
 
- interface an LPC4088 to a VGA monitor by leveraging the LCD controller and to use an accelerometer as the sensor for the player controller
 
- integrate two Zigbees RF modules integrated for wireless communication between the controller and LPC4088 board via UART protocol
 
- enable digital audio output by developing an I2S driver and porting an audio tracker library
 
- utilize the General Purpose Direct Memory Access (GPDMA) controller memory-to-memory function to decrease memory write times when updating the video buffer
 
- utilize the DMA controller's memory-to-peripheral function to help reduce frequent I2S interrupts when loading digital audio contents
 
- implement an SD card peripheral driver to facilitate memory exchange between an SD card

=== Team Members & Responsibilities ===
[[File:Cmpe-244-brian-pic.png ‎]]
* '''Brian Ho'''
** Wiki project schedule planning
** Interfacing the accelerometer
** Digital to Analog Conversion resistor ladder design used to interface digital LCD controller signal with analog VGA port signal
** PCB planning and acquisition
** Procurement of controller hardware
** Wireless communication via Zigbees RF modules and UART protocol
** Wireless controller logic and integration
** Ball movement and wall/brick collision logic
** Hardware/software development and integration
** Final testing and integration
[[File:Cmpe-244-billy-pic.png]]
* '''[https://www.linkedin.com/in/william-lai-525681139 Billy Lai]'''
** Wiki project schedule planning
** Increased CPU clock rate from 96 MHz to 120 MHz
** Initial memory write timing and measurements to estimate memory write capacity
** DMA driver memory-to-memory implementation for transferring graphics data to video buffer
** I2S with DMA memory-to-peripheral implementation to reduce memory-write interrupts when I2S Tx FIFO level is low
** Paddle movement and paddle collision logic, testing and integration
** Software development and integration
** Final testing and integration
[[File:Cmpe-244-jasdip-pic.png]]
* '''Jasdip Sekhon'''
** Wiki project schedule planning
** PCB design and planning
** File loading method algorithm for storing loaded files in memory
** Audio track and audio effects selection for gameplay audio
** Palette manipulation for gameplay
** Software development and integration
** Final testing and integration
[[File:Cmpe-244-isaac-pic.png ]]
* '''Isaac Wahhab'''
** Wiki project schedule planning
** DAC resistor ladder design and integration
** Hardware procurement (LPC4088 board, Zigbees, VGA monitor, etc)
** VGA driving through LCD controller
** Audio tracker library port
** Digital audio output using I2S
** Palette scrolling feature implementation
** Graphics rendering
** Vertical scrolling implementation for the video buffer
** Sprite testing and implementation for the ball
** SD card peripheral driver for reading assets from SD card
** Art assets acquisition
** Hardware/software development and integration
** Gameplay logic design
** Gameplay audio and level design
** Final testing and integration

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
*09/21/2021
|
*09/27/2021
|
* Read previous projects, gather information and discuss among the group members.
* Initial ordering of parts (LPC 4088 board, 64x64 LED screen)
|
* Completed
* Completed
|-
! scope="row"| 2
|
*09/28/2021
|
*10/04/2021
|
* Submit project proposals
|
* Completed
|-
! scope="row"| 3
|
*10/05/2021
|
*10/11/2021
|
* Test if board can drive VGA
* Test varying clock rates
* Hardware cursor on VGA
* Rough display driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 4
|
* 10/12/2021
|
* 10/18/2021
|
* DAC resistor ladder
* RGB channels
* Palette shifting
* Measure memory buffer writing timing
* Wiki schedule planning
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
*10/19/2021
|
*10/25/2021
|
* Calculate upper-bound for VGA refresh timings
* Create GitLab repository for project
* Joystick functionality
* SD card driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
*10/26/2021
|
*11/01/2021
|
* Audio peripheral
* Wireless communication using Zigbee module
* PCB planning
* Accelerometer driver
* Continue SD card driver
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/02/2021

|
* 11/08/2021
|
* Continue audio peripheral
* Continue wireless communication using Zigbee module
* Continue accelerometer driver
* PCB finalizing and ordering
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 11/09/2021
|
* 11/15/2021
|
* Convert PNG data to graphics format
* Game background functionality
* Sprite functionality
* User interface
* Integrate components
* Initial testing
|
* Completed
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 11/16/2021
|
* 11/22/2021
|
* Continue everything from last week
* Game physics (wall collision and AI movement)
* Measurements for wooden case
* Art assets (audio and visual)
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 10
|
*11/23/2021
|
*11/29/2021
|

* Fine-tune user experience
* Make case
|
* Completed
* Completed
|-
! scope="row"| 11
|
* 11/30/2021
|
* 12/06/2021
|
* Continue fine-tune user experience
* Integrate hardware
|
* Completed
* Completed
|-
! scope="row"| 12
|
* 12/07/2021
|
* 12/16/2021
|
* Final testing
* Final demo
|
* Completed
* Completed
|-
|}

== Bill Of Materials ==
{| class="wikitable" style="margin-left: 0px; margin-right: auto;"
|-
! scope="col"| Item#
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| [https://www.embeddedartists.com/products/lpc4088-developers-kit LPC4088-32 Developer's Kit]
| 1
| $300.00
|-
! scope="row"| 2
| Microcontroller Board
| [http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems/page/sj2-board SJ2 Board]
| 1
| $50.00
|-
! scope="row"| 3
| Bluetooth Module
| [https://hub.digi.com/support/products/digi-xbee/digi-xbee-s1 Digi XBee-S1]
| 2
| N/A
|-
! scope="row"| 4
| DAC PCB
| [https://jlcpcb.com JLCPCB (Set of 5)]
| 1
| $15.00
|-
! scope="row"| 4
| Monitor
| Asus TFT Monitor
| 1
| N/A
|-
! scope="row"| 5
| Mini Breadboard
| [https://www.amazon.com/EL-CK-002-Electronic-Breadboard-Capacitor-Potentiometer/dp/B01ERP6WL4/ref=pd_sbs_147_2/132-6936347-2117569?_encoding=UTF8&pd_rd_i=B01ERP6WL4&pd_rd_r=6a783095-ec87-42ae-9f01-a5ebc9a44607&pd_rd_w=V2aU6&pd_rd_wg=11h1i&pf_rd_p=d66372fe-68a6-48a3-90ec-41d7f64212be&pf_rd_r=368WNKXFBBQY2FJQ4119&psc=1&refRID=368WNKXFBBQY2FJQ4119 ELEGOO Mini Breadboard]
| 1
| $13.99
|-
! scope="row"| 6
| Arcade buttons
| [https://www.amazon.com/EG-STARTS-Illuminated-Buttons-Raspberry/dp/B01N11BDX9?pd_rd_w=DiSpt&pf_rd_p=5bc65336-a8b0-4888-9e70-4d4ce2f6bd59&pf_rd_r=JCH2VQYXFPHND4XZ4FAM&pd_rd_r=8d6243dd-e1bd-4dae-9b19-be2e00f3085f&pd_rd_wg=K4lt9&pd_rd_i=B01N11BDX9&psc=1&ref_=pd_bap_d_rp_1_i EG STARTS LED Arcade Buttons]
| 2
| $11.99
|-
! scope="row"| 7
| VGA cable
| N/A
| 1
| N/A
|-
! scope="row"| 7
| SD card
| N/A
| 1
| N/A
|-
|}

== Design & Implementation ==

=== Hardware Design ===
We designed our PCB using Autodesk EAGLE based on the tutorial on PCB design during the class lecture. We ordered our PCB to be manufactured from JLCPCB. The PCB consists of an R-2R resistor ladder DAC for the purpose of converting the digital signal from the microcontroller to analog to display graphics on the VGA monitor. Since the VGA input for each color channel is 0-0.7V, the resistor ladder scaled the LPC4088 outputs into proper DAC values for the VGA to take in. Five bits are passed to each of the red, green, and blue channels of the VGA in addition to other signals, such as Horizontal sync (HSYNC) and Vertical sync (VSYNC).
 
[[File:pcbSchematic.png|500px|thumb|left|PCB Schematic]]
[[File:pcbLayout.png|500px|thumb|right|PCB Layout]]
[[File:pcbPic-min.png|500px|thumb|right|Manufactured PCB]]
[[File:pcbManufactured.png|500px|thumb|left|Soldered PCB]]
 

=== Hardware Interface ===

Two Digi XBee devices were used to communicate between the SJ2-Board controller and the LPC4088 board to control the paddle on the game. The XBee devices were programmed using the XTCU programming application provided by Digi International. The SJ2-Board used one XBee as a transmitted and the LPC4088 used another as a receiver. The two modules communicated using UART frames and a single byte represented any change of movement or button presses sent by the controller.

{| class="wikitable"
|-
! scope="col"| XCTU
! scope="col"| Transmitter
! scope="col"| Receiver
|-
! scope="row"| CH
| C
| C
|-
! scope="row"| ID
| 6969
| 6969
|-
! scope="row"| CE
| Coordinator
| Endpoint
|-
! scope="row"| Baud Rate
| 9600
| 9600
|-
|}
[[File:sj2Board.png|380px|thumb|center|SJ2 Board]]
[[File:CMPE146_F16_MW_XbeeModule.jpg|200px|thumb|center|XBee Device]]
 

Two LED buttons were used to interface with the controller as inputs. They were both set up using GPIO.
[[File:VGA LED Button.jpg|200px|thumb|center|LED Button]]
 
[[File:VGA_Controller_Block_Diagram.PNG|380px|thumb|center|Controller Block Diagram]]
 

==== I2S Peripheral and Stereo Audio Decoder ====

The LPC4088 is equipped with an I2S stereo encoder. This module provides a hardware-enabled solution for sending I2S-encoded audio data packets to stereo decoders, which produce the corresponding audio waveform. For this project, since 16-bit stereo audio playback was chosen, data packets transmitted on this interface were sent in the following format:

[[File:I2S_audio_data_word.png]]

Here, each word of data encodes a single packet of audio in two 16-bit sections: one signal level for the left channel, and another for the right. These 4 bytes are sent at a rate of 44100 Hz, resulting in a total data transmission of 176400 bytes a second. Rather than needing to be software-defined, this transmission is automatically handled by hardware using a 8-word deep FIFO queue that manages streaming out the audio data at a fixed rate, saving a significant amount of CPU cycles.

This encoded audio signal was sent to a dedicated external [https://www.nxp.com/docs/en/data-sheet/UDA1380.pdf stereo decoder], which output to a 3.5 mm audio jack connected to speakers in the VGA monitor.

==== LCD Peripheral ====

The LCD peripheral was configured for 16-bit color palette (5 bits of color data for each color level -- red, green, and blue -- and an unused intensity bit), at a resolution of 800 by 600 pixels, and with an active color palette size of 256 colors. Since the logic for driving an LCD display has the same concepts (horizontal blanking, vertical blanking, etc) and timing information for any given resolution was the same, we were able to directly drive a VGA display with the same signals and timing data (after the digital signal had been converted to analog).

[[File:LCD_%2B_VGA_Video_signal.png]]

The timing register configurations needed to achieve the requested video display was derived from a dedicated website for [http://tinyvga.com/vga-timing/800x600@60Hz VGA signal info].

==== External Memory ====

The game's visual assets and tracker audio file (both after being loaded from an SD card) and the game's video buffer required more memory than what is present natively in the microcontroller. To accommodate this, an external 32MiB memory module was used. This module was interfaced with using the external memory controller peripheral present on the board.

=== Software Design ===
==== Controller ====

There is an internal accelerometer on the SJ2 board that is utilized as the movement controls for the paddle movement. The y-axis value of the magnitude is polled 17 times in 17 milliseconds and is divided by 17 to get an average reading per that time frame. The maximum tilt value that can be read by the controller is set to 700 out of the total 1000 (highest possible value). The current tilt value is divided by 31 to get a total of 32 possible readings to represent 5 bits of magnitude.

The controller sends one byte of data to the console whenever there is any change in magnitude on the accelerometer or button changes. The byte contains all the information needed to control the paddle on the screen. The first two bits on the byte represent 0 if there is no button input and 1 if there is a button input. The third bit represents whether the controller is tilted left or right with 0 being left and 1 being right. The last five bits represent the controller accelerometer's magnitude.

[[File:VGA_UART_Frame.PNG]]

==== Audio ====

Sound design was provided to the project (in the form of music and sound effects) through the use of a [https://https://en.wikipedia.org/wiki/Music_tracker tracker audio library]. Source code was adopted from the [https://github.com/schismtracker/schismtracker Schism Tracker] library and altered with the following modifications:

- All UI logic was stripped from the source code

- All file handling logic was reworked to use the SD card reading functions, as the project does not support POSIX file calls

- The [https://www.libsdl.org/ Simple DirectMedia Layer] hooks were replaced with a FreeRTOS task that periodically called the waveform generation callback function, which was passed to the I2S peripheral hooked up to a stereo decoder.

=== Implementation ===

==== Ball Logic ====

The ball is controlled by the buttons and the controller accelerometer values. When the button is pressed, the ball is initially sent with a random velocity. The velocity is determined by a substate timer and is initially set as one of 6 possible values.

{| class="wikitable"
|-
! scope="col"| Substate Count
! scope="col"| X-Velocity
! scope="col"| Y-Velocity
|-
! scope="row"| 0
| 4
| -2
|-
! scope="row"| 1
| -4
| -2
|-
! scope="row"| 2
| 2
| -4
|-
! scope="row"| 3
| -2
| -4
|-
! scope="row"| 4
| 1
| -5
|-
! scope="row"| 5
| -1
| -5
|-
|}

The ball's velocity changes whenever it detects a collision with either the side of the screen, the bricks on the top of the level, or the paddle at the bottom. If colliding with the paddle, the ball's velocity will take into account the paddle's velocity and add it to its current velocity. This is done by adding the paddle's x-velocity and adjusting the y-velocity by normalizing the velocity vectors to have a magnitude of 5. This means the ball will always be moving with a magnitude of 5 in any direction after it hits the paddle. If the ball collides with anything other than the paddle, then the velocity of the ball is changed with no increase or decrease of magnitude. If the ball collides on the side of a wall or block, the x-velocity will reflect (go from negative to positive or vice-versa) and the y-velocity will remain constant. If the ball collides on the top or bottom of a block, the y-velocity will reflect and the x-velocity will remain constant. If it appears that the ball collides with the side and top or bottom of something, then the ball will check whichever face it will collide with first and then calculate the other face. Lastly, when the ball collides with the bottom of the screen, then the player experiences a "death" and the ball will reset on the paddle. If the ball collides with the top of the screen, the screen will scroll up for the user to play on the next level.

Whenever the ball collides with a brick, the brick will disappear. To accomplish this, there is a 2-D array of unsigned integers that contains the values of the bricks. If the integer is 0, then there is no block. If the integer is any value other than 0, then the block is filled and has collision logic. If the ball approaches a region that might have a block, then there will be a check within that 2-D to see if the block is filled or empty. If the block is filled, there will be the collision logic that changes the vector of the ball and sets the integer value in the 2-D array to 0 to represent that the block has been broken.

==== Paddle Movement ====
The paddle movement is controlled by the accelerometer data being wirelessly transmitted from the SJ-Two board to the LPC4088 board. The accelerometer data will map to 63 possible indexes for the paddle location. Originally, the paddle would jump directly into the corresponding index location depending on the accelerometer input. We later iterated on this process by implementing an algorithm to smoothly transition into these indexes. This was done by calculating a paddle velocity using the "current desired paddle position" and the previous paddle position. We adjusted the parameters of the algorithm iteratively to produce the smooth paddle movement.

==== Paddle Collision Logic ====
The position of the ball is calculated by adding delta_x and delta_y to the ball's current x and y position. For each frame, if the ball's next Y position crosses the paddle's Y position, we would check for a paddle collision. To do this, we used the delta_x and delta_y to get the ball's movement as a slope. We then substituted the slope, the ball's current position, and 1 pixel above the paddle's Y position into a point-slope equation (y2 - y1) = m(x2 - x1) to get the paddle's X position at the time of the ball reaching the pixel above paddle's Y position. After finding the location where the ball is expected to collide with the paddle, we then calculated the ball's collision magnitude and the ball's next location's magnitude. Using a ratio of these two magnitudes, we have a frame time fraction of when the ball possibly collided with the paddle. Then, by using the paddle's current position and paddle velocity, we are able to find where the paddle previously was, and measure how far the paddle had moved during that time frame. By multiplying the time frame from the ball's collision magnitude to the paddle's change in X position, we are able to find whether the paddle was under the ball at the time of the ball reaching the pixel above the paddle's Y position.

== Testing & Technical Challenges ==

=== VGA Interfacing ===
A DAC resistor ladder was designed and a corresponding custom PCB was acquired to interface the 3.3V digital LCD controller signal with the analog VGA input signal. The VGA cable takes between 0V (completely dark) and 0.7V (maximum brightness) for color pin inputs. The DAC resistor ladder was designed and implemented on a breadboard to verify the design. Validating the design on a breadboard gave us a clear vision during PCB design and planning.

=== VGA Write Timings ===
There were initial tests that were required to be done to see if using a VGA monitor as an output was a viable option for this project. We had to test memory write timings to see if they were fast enough to match the refresh rate of the monitor. Those tests would decide how much of the screen buffer we could write to during the blanking periods during the VGA refresh. This was a good learning experience for testing hardware capabilities and should be done whenever introducing any new aspects to a project.

=== Ball Collision Logic ===
The initial version of the ball collision used pixel values to see if the ball was entering the brick. The problem was that this logic did not account for the case where the ball enters the brick diagonally and the logic for that version of the collision had to be scrapped. The later iteration of the logic was changed to checking whether the ball was potentially crossing into an area that had a brick. This was done by separating the checks into a grid that represented the locations of the blocks. In this case, there was an initial check to see if the ball was crossing a boundary where there might be a new brick. If the ball is crossing the x-boundary and not the y-boundary, then there would be a collision check to see if it collided with a block. If it is crossing the y-boundary and not the x-boundary, once again then there would be a collision check to see if it collided with a block. If it is crossing both the x-boundary and y-boundary, then there would be a check to see which boundary the ball would cross first. This new logic allowed for us to account for the case where the block enters the block diagonally. This taught us that sometimes it is beneficial to scrap a previous iteration and use a new approach to solving a problem because it may lead to better results.

=== Paddle Movement ===
One issue we discovered was the paddle would slow down as it approached the walls. This was due to the difference in "current desired paddle position" and the previous paddle position becoming smaller as the walls were approached, leading to a smaller paddle velocity. To account for this, whenever the accelerometer sent data that mapped to the minimum or maximum paddle index, we would increase the distance of the "current desired paddle position" so the paddle velocity would not slow the paddle down as it approached the walls.

=== Paddle Collision Logic ===
During final integration and testing, one of the biggest issues we encountered was a bug where the ball would not detect a collision with the paddle. The ball would phase right through the paddle instead of bouncing off of it. One of the great ideas that we thought of was to change the color of the ball during each check to see if the checks would go through. This debug check allowed us to verify that section of code that checked for paddle collisions was being skipped even though the ball's next Y position had crossed the paddle's Y position. By having this debugging method, we were able to root cause the issue.

== Conclusion ==
For some of us, this was our first experience in designing and implementing a functional embedded system using FreeRTOS. The challenges we hurdled to accomplish our technical goals helped to shape a journey that was greatly enhanced by the opportunity to meet in person and having a hands-on approach to hardware and component integration. We were able to utilize design principles and implement communication protocols we learned in lecture to pull this project together. Some useful topics include: UART, SPI, I2C, I2S, DMA, FreeRTOS mutexes and binary semaphores, queues, task creation, stack size, task scheduler, interrupt service routines, bootloaders, etc. After extensive hours spent testing and verifying our implementation, we have learned that time allocated for integration and testing will significantly improve chances of a successful project. If we had skipped out on testing and verification, the project could have possibly crashed and burned when it was delivered. We hope this project can be of use to any future engineers following behind us, and we wish you all good times and happy development ahead!

=== Project Video ===
[https://www.youtube.com/watch?v=ipngAGlN86c SJSU CMPE 244 FALL 2021: Embedded Software Project Gameplay]

=== Project Source Code ===
* [https://gitlab.com/fayek.wahhab/sjtwo-c/-/tree/feature/i2s_dma2 Treasure Dive Baseline for Demo Source Code]
* [https://gitlab.com/hobriandh/sjtwo-c/-/tree/final_controller Controller for Demo Source Code]

== References ==
=== Acknowledgement ===
We would like to acknowledge and thank our instructor Preetpal Kang for sharing his knowledge in embedded software development using FreeRTOS.
 
Another shoutout to our art designer Jacob for the unique artwork displayed in our game.

=== Appendix ===
*[https://gitlab.com/sjtwo-c-dev/sjtwo-c/-/blob/master/LPC408X_7X%20User%20Manual.pdf LPC408x/407x Datasheet]
*[https://www.freertos.org/Documentation/RTOS_book.html FreeRTOS Documentation]
*[http://elm-chan.org/fsw/ff/00index_e.html FatFs - Generic FAT Filesystem Module]
*[https://www.digi.com/resources/library/data-sheets/ds_xbeemultipointmodules Xbee Datasheet]
*[http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems Embedded Drivers & Real Time Operating Systems Lessons]

F21: Treasure Dive

2021-12-18T12:53:28Z

Proj user1: /* Team Members & Responsibilities */

== '''Treasure Dive''' ==
[[File:mermaid.png|thumb|500px|right|Game Screen Capture]]
[[File:Cmpe-244-controller-pic.png|thumb|600px|right|Game Controller]]

== Abstract ==
Treasure Dive is a modern one player game inspired by the popular arcade game Breakout, which was released by Atari in 1976. The ancients rambled about an unconfirmed tale of a shipwreck that saw no survivors. As an adventurous and skilled free diver, you are seeking treasure buried deep in an ocean abyss. You dove down all the way to the seafloor and found the buried treasure but need to make it back to the surface to see another day! The player uses a wireless controller to control an on-screen paddle to break bricks on the top of the screen. When enough bricks are broken, a passage is revealed, allowing you to get closer to the surface.

== Objectives & Introduction ==
The objective of this project is to:
 
- interface an LPC4088 to a VGA monitor by leveraging the LCD controller and to use an accelerometer as the sensor for the player controller
 
- integrate two Zigbees RF modules integrated for wireless communication between the controller and LPC4088 board via UART protocol
 
- enable digital audio output by developing an I2S driver and porting an audio tracker library
 
- utilize the General Purpose Direct Memory Access (GPDMA) controller memory-to-memory function to decrease memory write times when updating the video buffer
 
- utilize the DMA controller's memory-to-peripheral function to help reduce frequent I2S interrupts when loading digital audio contents
 
- implement an SD card peripheral driver to facilitate memory exchange between an SD card

=== Team Members & Responsibilities ===
[[File:Cmpe-244-brian-pic.png ‎]]
'''Brian Ho'''
** Wiki project schedule planning
** Interfacing the accelerometer
** Digital to Analog Conversion resistor ladder design used to interface digital LCD controller signal with analog VGA port signal
** PCB planning and acquisition
** Procurement of controller hardware
** Wireless communication via Zigbees RF modules and UART protocol
** Wireless controller logic and integration
** Ball movement and wall/brick collision logic
** Hardware/software development and integration
** Final testing and integration
[[File:Cmpe-244-billy-pic.png]]
'''[https://www.linkedin.com/in/william-lai-525681139 Billy Lai]'''
** Wiki project schedule planning
** Increased CPU clock rate from 96 MHz to 120 MHz
** Initial memory write timing and measurements to estimate memory write capacity
** DMA driver memory-to-memory implementation for transferring graphics data to video buffer
** I2S with DMA memory-to-peripheral implementation to reduce memory-write interrupts when I2S Tx FIFO level is low
** Paddle movement and paddle collision logic, testing and integration
** Software development and integration
** Final testing and integration
[[File:Cmpe-244-jasdip-pic.png]]
'''Jasdip Sekhon'''
** Wiki project schedule planning
** PCB design and planning
** File loading method algorithm for storing loaded files in memory
** Audio track and audio effects selection for gameplay audio
** Palette manipulation for gameplay
** Software development and integration
** Final testing and integration
[[File:Cmpe-244-isaac-pic.png ]]
'''Isaac Wahhab'''
** Wiki project schedule planning
** DAC resistor ladder design and integration
** Hardware procurement (LPC4088 board, Zigbees, VGA monitor, etc)
** VGA driving through LCD controller
** Audio tracker library port
** Digital audio output using I2S
** Palette scrolling feature implementation
** Graphics rendering
** Vertical scrolling implementation for the video buffer
** Sprite testing and implementation for the ball
** SD card peripheral driver for reading assets from SD card
** Art assets acquisition
** Hardware/software development and integration
** Gameplay logic design
** Gameplay audio and level design
** Final testing and integration

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
*09/21/2021
|
*09/27/2021
|
* Read previous projects, gather information and discuss among the group members.
* Initial ordering of parts (LPC 4088 board, 64x64 LED screen)
|
* Completed
* Completed
|-
! scope="row"| 2
|
*09/28/2021
|
*10/04/2021
|
* Submit project proposals
|
* Completed
|-
! scope="row"| 3
|
*10/05/2021
|
*10/11/2021
|
* Test if board can drive VGA
* Test varying clock rates
* Hardware cursor on VGA
* Rough display driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 4
|
* 10/12/2021
|
* 10/18/2021
|
* DAC resistor ladder
* RGB channels
* Palette shifting
* Measure memory buffer writing timing
* Wiki schedule planning
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
*10/19/2021
|
*10/25/2021
|
* Calculate upper-bound for VGA refresh timings
* Create GitLab repository for project
* Joystick functionality
* SD card driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
*10/26/2021
|
*11/01/2021
|
* Audio peripheral
* Wireless communication using Zigbee module
* PCB planning
* Accelerometer driver
* Continue SD card driver
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/02/2021

|
* 11/08/2021
|
* Continue audio peripheral
* Continue wireless communication using Zigbee module
* Continue accelerometer driver
* PCB finalizing and ordering
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 11/09/2021
|
* 11/15/2021
|
* Convert PNG data to graphics format
* Game background functionality
* Sprite functionality
* User interface
* Integrate components
* Initial testing
|
* Completed
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 11/16/2021
|
* 11/22/2021
|
* Continue everything from last week
* Game physics (wall collision and AI movement)
* Measurements for wooden case
* Art assets (audio and visual)
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 10
|
*11/23/2021
|
*11/29/2021
|

* Fine-tune user experience
* Make case
|
* Completed
* Completed
|-
! scope="row"| 11
|
* 11/30/2021
|
* 12/06/2021
|
* Continue fine-tune user experience
* Integrate hardware
|
* Completed
* Completed
|-
! scope="row"| 12
|
* 12/07/2021
|
* 12/16/2021
|
* Final testing
* Final demo
|
* Completed
* Completed
|-
|}

== Bill Of Materials ==
{| class="wikitable" style="margin-left: 0px; margin-right: auto;"
|-
! scope="col"| Item#
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| [https://www.embeddedartists.com/products/lpc4088-developers-kit LPC4088-32 Developer's Kit]
| 1
| $300.00
|-
! scope="row"| 2
| Microcontroller Board
| [http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems/page/sj2-board SJ2 Board]
| 1
| $50.00
|-
! scope="row"| 3
| Bluetooth Module
| [https://hub.digi.com/support/products/digi-xbee/digi-xbee-s1 Digi XBee-S1]
| 2
| N/A
|-
! scope="row"| 4
| DAC PCB
| [https://jlcpcb.com JLCPCB (Set of 5)]
| 1
| $15.00
|-
! scope="row"| 4
| Monitor
| Asus TFT Monitor
| 1
| N/A
|-
! scope="row"| 5
| Mini Breadboard
| [https://www.amazon.com/EL-CK-002-Electronic-Breadboard-Capacitor-Potentiometer/dp/B01ERP6WL4/ref=pd_sbs_147_2/132-6936347-2117569?_encoding=UTF8&pd_rd_i=B01ERP6WL4&pd_rd_r=6a783095-ec87-42ae-9f01-a5ebc9a44607&pd_rd_w=V2aU6&pd_rd_wg=11h1i&pf_rd_p=d66372fe-68a6-48a3-90ec-41d7f64212be&pf_rd_r=368WNKXFBBQY2FJQ4119&psc=1&refRID=368WNKXFBBQY2FJQ4119 ELEGOO Mini Breadboard]
| 1
| $13.99
|-
! scope="row"| 6
| Arcade buttons
| [https://www.amazon.com/EG-STARTS-Illuminated-Buttons-Raspberry/dp/B01N11BDX9?pd_rd_w=DiSpt&pf_rd_p=5bc65336-a8b0-4888-9e70-4d4ce2f6bd59&pf_rd_r=JCH2VQYXFPHND4XZ4FAM&pd_rd_r=8d6243dd-e1bd-4dae-9b19-be2e00f3085f&pd_rd_wg=K4lt9&pd_rd_i=B01N11BDX9&psc=1&ref_=pd_bap_d_rp_1_i EG STARTS LED Arcade Buttons]
| 2
| $11.99
|-
! scope="row"| 7
| VGA cable
| N/A
| 1
| N/A
|-
! scope="row"| 7
| SD card
| N/A
| 1
| N/A
|-
|}

== Design & Implementation ==

=== Hardware Design ===
We designed our PCB using Autodesk EAGLE based on the tutorial on PCB design during the class lecture. We ordered our PCB to be manufactured from JLCPCB. The PCB consists of an R-2R resistor ladder DAC for the purpose of converting the digital signal from the microcontroller to analog to display graphics on the VGA monitor. Since the VGA input for each color channel is 0-0.7V, the resistor ladder scaled the LPC4088 outputs into proper DAC values for the VGA to take in. Five bits are passed to each of the red, green, and blue channels of the VGA in addition to other signals, such as Horizontal sync (HSYNC) and Vertical sync (VSYNC).
 
[[File:pcbSchematic.png|500px|thumb|left|PCB Schematic]]
[[File:pcbLayout.png|500px|thumb|right|PCB Layout]]
[[File:pcbPic-min.png|500px|thumb|right|Manufactured PCB]]
[[File:pcbManufactured.png|500px|thumb|left|Soldered PCB]]
 

=== Hardware Interface ===

Two Digi XBee devices were used to communicate between the SJ2-Board controller and the LPC4088 board to control the paddle on the game. The XBee devices were programmed using the XTCU programming application provided by Digi International. The SJ2-Board used one XBee as a transmitted and the LPC4088 used another as a receiver. The two modules communicated using UART frames and a single byte represented any change of movement or button presses sent by the controller.

{| class="wikitable"
|-
! scope="col"| XCTU
! scope="col"| Transmitter
! scope="col"| Receiver
|-
! scope="row"| CH
| C
| C
|-
! scope="row"| ID
| 6969
| 6969
|-
! scope="row"| CE
| Coordinator
| Endpoint
|-
! scope="row"| Baud Rate
| 9600
| 9600
|-
|}
[[File:sj2Board.png|380px|thumb|center|SJ2 Board]]
[[File:CMPE146_F16_MW_XbeeModule.jpg|200px|thumb|center|XBee Device]]
 

Two LED buttons were used to interface with the controller as inputs. They were both set up using GPIO.
[[File:VGA LED Button.jpg|200px|thumb|center|LED Button]]
 
[[File:VGA_Controller_Block_Diagram.PNG|380px|thumb|center|Controller Block Diagram]]
 

==== I2S Peripheral and Stereo Audio Decoder ====

The LPC4088 is equipped with an I2S stereo encoder. This module provides a hardware-enabled solution for sending I2S-encoded audio data packets to stereo decoders, which produce the corresponding audio waveform. For this project, since 16-bit stereo audio playback was chosen, data packets transmitted on this interface were sent in the following format:

[[File:I2S_audio_data_word.png]]

Here, each word of data encodes a single packet of audio in two 16-bit sections: one signal level for the left channel, and another for the right. These 4 bytes are sent at a rate of 44100 Hz, resulting in a total data transmission of 176400 bytes a second. Rather than needing to be software-defined, this transmission is automatically handled by hardware using a 8-word deep FIFO queue that manages streaming out the audio data at a fixed rate, saving a significant amount of CPU cycles.

This encoded audio signal was sent to a dedicated external [https://www.nxp.com/docs/en/data-sheet/UDA1380.pdf stereo decoder], which output to a 3.5 mm audio jack connected to speakers in the VGA monitor.

==== LCD Peripheral ====

The LCD peripheral was configured for 16-bit color palette (5 bits of color data for each color level -- red, green, and blue -- and an unused intensity bit), at a resolution of 800 by 600 pixels, and with an active color palette size of 256 colors. Since the logic for driving an LCD display has the same concepts (horizontal blanking, vertical blanking, etc) and timing information for any given resolution was the same, we were able to directly drive a VGA display with the same signals and timing data (after the digital signal had been converted to analog).

[[File:LCD_%2B_VGA_Video_signal.png]]

The timing register configurations needed to achieve the requested video display was derived from a dedicated website for [http://tinyvga.com/vga-timing/800x600@60Hz VGA signal info].

==== External Memory ====

The game's visual assets and tracker audio file (both after being loaded from an SD card) and the game's video buffer required more memory than what is present natively in the microcontroller. To accommodate this, an external 32MiB memory module was used. This module was interfaced with using the external memory controller peripheral present on the board.

=== Software Design ===
==== Controller ====

There is an internal accelerometer on the SJ2 board that is utilized as the movement controls for the paddle movement. The y-axis value of the magnitude is polled 17 times in 17 milliseconds and is divided by 17 to get an average reading per that time frame. The maximum tilt value that can be read by the controller is set to 700 out of the total 1000 (highest possible value). The current tilt value is divided by 31 to get a total of 32 possible readings to represent 5 bits of magnitude.

The controller sends one byte of data to the console whenever there is any change in magnitude on the accelerometer or button changes. The byte contains all the information needed to control the paddle on the screen. The first two bits on the byte represent 0 if there is no button input and 1 if there is a button input. The third bit represents whether the controller is tilted left or right with 0 being left and 1 being right. The last five bits represent the controller accelerometer's magnitude.

[[File:VGA_UART_Frame.PNG]]

==== Audio ====

Sound design was provided to the project (in the form of music and sound effects) through the use of a [https://https://en.wikipedia.org/wiki/Music_tracker tracker audio library]. Source code was adopted from the [https://github.com/schismtracker/schismtracker Schism Tracker] library and altered with the following modifications:

- All UI logic was stripped from the source code

- All file handling logic was reworked to use the SD card reading functions, as the project does not support POSIX file calls

- The [https://www.libsdl.org/ Simple DirectMedia Layer] hooks were replaced with a FreeRTOS task that periodically called the waveform generation callback function, which was passed to the I2S peripheral hooked up to a stereo decoder.

=== Implementation ===

==== Ball Logic ====

The ball is controlled by the buttons and the controller accelerometer values. When the button is pressed, the ball is initially sent with a random velocity. The velocity is determined by a substate timer and is initially set as one of 6 possible values.

{| class="wikitable"
|-
! scope="col"| Substate Count
! scope="col"| X-Velocity
! scope="col"| Y-Velocity
|-
! scope="row"| 0
| 4
| -2
|-
! scope="row"| 1
| -4
| -2
|-
! scope="row"| 2
| 2
| -4
|-
! scope="row"| 3
| -2
| -4
|-
! scope="row"| 4
| 1
| -5
|-
! scope="row"| 5
| -1
| -5
|-
|}

The ball's velocity changes whenever it detects a collision with either the side of the screen, the bricks on the top of the level, or the paddle at the bottom. If colliding with the paddle, the ball's velocity will take into account the paddle's velocity and add it to its current velocity. This is done by adding the paddle's x-velocity and adjusting the y-velocity by normalizing the velocity vectors to have a magnitude of 5. This means the ball will always be moving with a magnitude of 5 in any direction after it hits the paddle. If the ball collides with anything other than the paddle, then the velocity of the ball is changed with no increase or decrease of magnitude. If the ball collides on the side of a wall or block, the x-velocity will reflect (go from negative to positive or vice-versa) and the y-velocity will remain constant. If the ball collides on the top or bottom of a block, the y-velocity will reflect and the x-velocity will remain constant. If it appears that the ball collides with the side and top or bottom of something, then the ball will check whichever face it will collide with first and then calculate the other face. Lastly, when the ball collides with the bottom of the screen, then the player experiences a "death" and the ball will reset on the paddle. If the ball collides with the top of the screen, the screen will scroll up for the user to play on the next level.

Whenever the ball collides with a brick, the brick will disappear. To accomplish this, there is a 2-D array of unsigned integers that contains the values of the bricks. If the integer is 0, then there is no block. If the integer is any value other than 0, then the block is filled and has collision logic. If the ball approaches a region that might have a block, then there will be a check within that 2-D to see if the block is filled or empty. If the block is filled, there will be the collision logic that changes the vector of the ball and sets the integer value in the 2-D array to 0 to represent that the block has been broken.

==== Paddle Movement ====
The paddle movement is controlled by the accelerometer data being wirelessly transmitted from the SJ-Two board to the LPC4088 board. The accelerometer data will map to 63 possible indexes for the paddle location. Originally, the paddle would jump directly into the corresponding index location depending on the accelerometer input. We later iterated on this process by implementing an algorithm to smoothly transition into these indexes. This was done by calculating a paddle velocity using the "current desired paddle position" and the previous paddle position. We adjusted the parameters of the algorithm iteratively to produce the smooth paddle movement.

==== Paddle Collision Logic ====
The position of the ball is calculated by adding delta_x and delta_y to the ball's current x and y position. For each frame, if the ball's next Y position crosses the paddle's Y position, we would check for a paddle collision. To do this, we used the delta_x and delta_y to get the ball's movement as a slope. We then substituted the slope, the ball's current position, and 1 pixel above the paddle's Y position into a point-slope equation (y2 - y1) = m(x2 - x1) to get the paddle's X position at the time of the ball reaching the pixel above paddle's Y position. After finding the location where the ball is expected to collide with the paddle, we then calculated the ball's collision magnitude and the ball's next location's magnitude. Using a ratio of these two magnitudes, we have a frame time fraction of when the ball possibly collided with the paddle. Then, by using the paddle's current position and paddle velocity, we are able to find where the paddle previously was, and measure how far the paddle had moved during that time frame. By multiplying the time frame from the ball's collision magnitude to the paddle's change in X position, we are able to find whether the paddle was under the ball at the time of the ball reaching the pixel above the paddle's Y position.

== Testing & Technical Challenges ==

=== VGA Interfacing ===
A DAC resistor ladder was designed and a corresponding custom PCB was acquired to interface the 3.3V digital LCD controller signal with the analog VGA input signal. The VGA cable takes between 0V (completely dark) and 0.7V (maximum brightness) for color pin inputs. The DAC resistor ladder was designed and implemented on a breadboard to verify the design. Validating the design on a breadboard gave us a clear vision during PCB design and planning.

=== VGA Write Timings ===
There were initial tests that were required to be done to see if using a VGA monitor as an output was a viable option for this project. We had to test memory write timings to see if they were fast enough to match the refresh rate of the monitor. Those tests would decide how much of the screen buffer we could write to during the blanking periods during the VGA refresh. This was a good learning experience for testing hardware capabilities and should be done whenever introducing any new aspects to a project.

=== Ball Collision Logic ===
The initial version of the ball collision used pixel values to see if the ball was entering the brick. The problem was that this logic did not account for the case where the ball enters the brick diagonally and the logic for that version of the collision had to be scrapped. The later iteration of the logic was changed to checking whether the ball was potentially crossing into an area that had a brick. This was done by separating the checks into a grid that represented the locations of the blocks. In this case, there was an initial check to see if the ball was crossing a boundary where there might be a new brick. If the ball is crossing the x-boundary and not the y-boundary, then there would be a collision check to see if it collided with a block. If it is crossing the y-boundary and not the x-boundary, once again then there would be a collision check to see if it collided with a block. If it is crossing both the x-boundary and y-boundary, then there would be a check to see which boundary the ball would cross first. This new logic allowed for us to account for the case where the block enters the block diagonally. This taught us that sometimes it is beneficial to scrap a previous iteration and use a new approach to solving a problem because it may lead to better results.

=== Paddle Movement ===
One issue we discovered was the paddle would slow down as it approached the walls. This was due to the difference in "current desired paddle position" and the previous paddle position becoming smaller as the walls were approached, leading to a smaller paddle velocity. To account for this, whenever the accelerometer sent data that mapped to the minimum or maximum paddle index, we would increase the distance of the "current desired paddle position" so the paddle velocity would not slow the paddle down as it approached the walls.

=== Paddle Collision Logic ===
During final integration and testing, one of the biggest issues we encountered was a bug where the ball would not detect a collision with the paddle. The ball would phase right through the paddle instead of bouncing off of it. One of the great ideas that we thought of was to change the color of the ball during each check to see if the checks would go through. This debug check allowed us to verify that section of code that checked for paddle collisions was being skipped even though the ball's next Y position had crossed the paddle's Y position. By having this debugging method, we were able to root cause the issue.

== Conclusion ==
For some of us, this was our first experience in designing and implementing a functional embedded system using FreeRTOS. The challenges we hurdled to accomplish our technical goals helped to shape a journey that was greatly enhanced by the opportunity to meet in person and having a hands-on approach to hardware and component integration. We were able to utilize design principles and implement communication protocols we learned in lecture to pull this project together. Some useful topics include: UART, SPI, I2C, I2S, DMA, FreeRTOS mutexes and binary semaphores, queues, task creation, stack size, task scheduler, interrupt service routines, bootloaders, etc. After extensive hours spent testing and verifying our implementation, we have learned that time allocated for integration and testing will significantly improve chances of a successful project. If we had skipped out on testing and verification, the project could have possibly crashed and burned when it was delivered. We hope this project can be of use to any future engineers following behind us, and we wish you all good times and happy development ahead!

=== Project Video ===
[https://www.youtube.com/watch?v=ipngAGlN86c SJSU CMPE 244 FALL 2021: Embedded Software Project Gameplay]

=== Project Source Code ===
* [https://gitlab.com/fayek.wahhab/sjtwo-c/-/tree/feature/i2s_dma2 Treasure Dive Baseline for Demo Source Code]
* [https://gitlab.com/hobriandh/sjtwo-c/-/tree/final_controller Controller for Demo Source Code]

== References ==
=== Acknowledgement ===
We would like to acknowledge and thank our instructor Preetpal Kang for sharing his knowledge in embedded software development using FreeRTOS.
 
Another shoutout to our art designer Jacob for the unique artwork displayed in our game.

=== Appendix ===
*[https://gitlab.com/sjtwo-c-dev/sjtwo-c/-/blob/master/LPC408X_7X%20User%20Manual.pdf LPC408x/407x Datasheet]
*[https://www.freertos.org/Documentation/RTOS_book.html FreeRTOS Documentation]
*[http://elm-chan.org/fsw/ff/00index_e.html FatFs - Generic FAT Filesystem Module]
*[https://www.digi.com/resources/library/data-sheets/ds_xbeemultipointmodules Xbee Datasheet]
*[http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems Embedded Drivers & Real Time Operating Systems Lessons]

F21: Treasure Dive

2021-12-18T12:52:45Z

Proj user1: /* Team Members & Responsibilities */

== '''Treasure Dive''' ==
[[File:mermaid.png|thumb|500px|right|Game Screen Capture]]
[[File:Cmpe-244-controller-pic.png|thumb|600px|right|Game Controller]]

== Abstract ==
Treasure Dive is a modern one player game inspired by the popular arcade game Breakout, which was released by Atari in 1976. The ancients rambled about an unconfirmed tale of a shipwreck that saw no survivors. As an adventurous and skilled free diver, you are seeking treasure buried deep in an ocean abyss. You dove down all the way to the seafloor and found the buried treasure but need to make it back to the surface to see another day! The player uses a wireless controller to control an on-screen paddle to break bricks on the top of the screen. When enough bricks are broken, a passage is revealed, allowing you to get closer to the surface.

== Objectives & Introduction ==
The objective of this project is to:
 
- interface an LPC4088 to a VGA monitor by leveraging the LCD controller and to use an accelerometer as the sensor for the player controller
 
- integrate two Zigbees RF modules integrated for wireless communication between the controller and LPC4088 board via UART protocol
 
- enable digital audio output by developing an I2S driver and porting an audio tracker library
 
- utilize the General Purpose Direct Memory Access (GPDMA) controller memory-to-memory function to decrease memory write times when updating the video buffer
 
- utilize the DMA controller's memory-to-peripheral function to help reduce frequent I2S interrupts when loading digital audio contents
 
- implement an SD card peripheral driver to facilitate memory exchange between an SD card

=== Team Members & Responsibilities ===
[[File:Cmpe-244-brian-pic.png ‎]]
'''* Brian Ho'''
** Wiki project schedule planning
** Interfacing the accelerometer
** Digital to Analog Conversion resistor ladder design used to interface digital LCD controller signal with analog VGA port signal
** PCB planning and acquisition
** Procurement of controller hardware
** Wireless communication via Zigbees RF modules and UART protocol
** Wireless controller logic and integration
** Ball movement and wall/brick collision logic
** Hardware/software development and integration
** Final testing and integration
[[File:Cmpe-244-billy-pic.png]]
'''* [https://www.linkedin.com/in/william-lai-525681139 Billy Lai]'''
** Wiki project schedule planning
** Increased CPU clock rate from 96 MHz to 120 MHz
** Initial memory write timing and measurements to estimate memory write capacity
** DMA driver memory-to-memory implementation for transferring graphics data to video buffer
** I2S with DMA memory-to-peripheral implementation to reduce memory-write interrupts when I2S Tx FIFO level is low
** Paddle movement and paddle collision logic, testing and integration
** Software development and integration
** Final testing and integration
[[File:Cmpe-244-jasdip-pic.png]]
'''* Jasdip Sekhon'''
** Wiki project schedule planning
** PCB design and planning
** File loading method algorithm for storing loaded files in memory
** Audio track and audio effects selection for gameplay audio
** Palette manipulation for gameplay
** Software development and integration
** Final testing and integration
[[File:Cmpe-244-isaac-pic.png ]]
'''* Isaac Wahhab'''
** Wiki project schedule planning
** DAC resistor ladder design and integration
** Hardware procurement (LPC4088 board, Zigbees, VGA monitor, etc)
** VGA driving through LCD controller
** Audio tracker library port
** Digital audio output using I2S
** Palette scrolling feature implementation
** Graphics rendering
** Vertical scrolling implementation for the video buffer
** Sprite testing and implementation for the ball
** SD card peripheral driver for reading assets from SD card
** Art assets acquisition
** Hardware/software development and integration
** Gameplay logic design
** Gameplay audio and level design
** Final testing and integration

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
*09/21/2021
|
*09/27/2021
|
* Read previous projects, gather information and discuss among the group members.
* Initial ordering of parts (LPC 4088 board, 64x64 LED screen)
|
* Completed
* Completed
|-
! scope="row"| 2
|
*09/28/2021
|
*10/04/2021
|
* Submit project proposals
|
* Completed
|-
! scope="row"| 3
|
*10/05/2021
|
*10/11/2021
|
* Test if board can drive VGA
* Test varying clock rates
* Hardware cursor on VGA
* Rough display driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 4
|
* 10/12/2021
|
* 10/18/2021
|
* DAC resistor ladder
* RGB channels
* Palette shifting
* Measure memory buffer writing timing
* Wiki schedule planning
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
*10/19/2021
|
*10/25/2021
|
* Calculate upper-bound for VGA refresh timings
* Create GitLab repository for project
* Joystick functionality
* SD card driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
*10/26/2021
|
*11/01/2021
|
* Audio peripheral
* Wireless communication using Zigbee module
* PCB planning
* Accelerometer driver
* Continue SD card driver
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/02/2021

|
* 11/08/2021
|
* Continue audio peripheral
* Continue wireless communication using Zigbee module
* Continue accelerometer driver
* PCB finalizing and ordering
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 11/09/2021
|
* 11/15/2021
|
* Convert PNG data to graphics format
* Game background functionality
* Sprite functionality
* User interface
* Integrate components
* Initial testing
|
* Completed
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 11/16/2021
|
* 11/22/2021
|
* Continue everything from last week
* Game physics (wall collision and AI movement)
* Measurements for wooden case
* Art assets (audio and visual)
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 10
|
*11/23/2021
|
*11/29/2021
|

* Fine-tune user experience
* Make case
|
* Completed
* Completed
|-
! scope="row"| 11
|
* 11/30/2021
|
* 12/06/2021
|
* Continue fine-tune user experience
* Integrate hardware
|
* Completed
* Completed
|-
! scope="row"| 12
|
* 12/07/2021
|
* 12/16/2021
|
* Final testing
* Final demo
|
* Completed
* Completed
|-
|}

== Bill Of Materials ==
{| class="wikitable" style="margin-left: 0px; margin-right: auto;"
|-
! scope="col"| Item#
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| [https://www.embeddedartists.com/products/lpc4088-developers-kit LPC4088-32 Developer's Kit]
| 1
| $300.00
|-
! scope="row"| 2
| Microcontroller Board
| [http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems/page/sj2-board SJ2 Board]
| 1
| $50.00
|-
! scope="row"| 3
| Bluetooth Module
| [https://hub.digi.com/support/products/digi-xbee/digi-xbee-s1 Digi XBee-S1]
| 2
| N/A
|-
! scope="row"| 4
| DAC PCB
| [https://jlcpcb.com JLCPCB (Set of 5)]
| 1
| $15.00
|-
! scope="row"| 4
| Monitor
| Asus TFT Monitor
| 1
| N/A
|-
! scope="row"| 5
| Mini Breadboard
| [https://www.amazon.com/EL-CK-002-Electronic-Breadboard-Capacitor-Potentiometer/dp/B01ERP6WL4/ref=pd_sbs_147_2/132-6936347-2117569?_encoding=UTF8&pd_rd_i=B01ERP6WL4&pd_rd_r=6a783095-ec87-42ae-9f01-a5ebc9a44607&pd_rd_w=V2aU6&pd_rd_wg=11h1i&pf_rd_p=d66372fe-68a6-48a3-90ec-41d7f64212be&pf_rd_r=368WNKXFBBQY2FJQ4119&psc=1&refRID=368WNKXFBBQY2FJQ4119 ELEGOO Mini Breadboard]
| 1
| $13.99
|-
! scope="row"| 6
| Arcade buttons
| [https://www.amazon.com/EG-STARTS-Illuminated-Buttons-Raspberry/dp/B01N11BDX9?pd_rd_w=DiSpt&pf_rd_p=5bc65336-a8b0-4888-9e70-4d4ce2f6bd59&pf_rd_r=JCH2VQYXFPHND4XZ4FAM&pd_rd_r=8d6243dd-e1bd-4dae-9b19-be2e00f3085f&pd_rd_wg=K4lt9&pd_rd_i=B01N11BDX9&psc=1&ref_=pd_bap_d_rp_1_i EG STARTS LED Arcade Buttons]
| 2
| $11.99
|-
! scope="row"| 7
| VGA cable
| N/A
| 1
| N/A
|-
! scope="row"| 7
| SD card
| N/A
| 1
| N/A
|-
|}

== Design & Implementation ==

=== Hardware Design ===
We designed our PCB using Autodesk EAGLE based on the tutorial on PCB design during the class lecture. We ordered our PCB to be manufactured from JLCPCB. The PCB consists of an R-2R resistor ladder DAC for the purpose of converting the digital signal from the microcontroller to analog to display graphics on the VGA monitor. Since the VGA input for each color channel is 0-0.7V, the resistor ladder scaled the LPC4088 outputs into proper DAC values for the VGA to take in. Five bits are passed to each of the red, green, and blue channels of the VGA in addition to other signals, such as Horizontal sync (HSYNC) and Vertical sync (VSYNC).
 
[[File:pcbSchematic.png|500px|thumb|left|PCB Schematic]]
[[File:pcbLayout.png|500px|thumb|right|PCB Layout]]
[[File:pcbPic-min.png|500px|thumb|right|Manufactured PCB]]
[[File:pcbManufactured.png|500px|thumb|left|Soldered PCB]]
 

=== Hardware Interface ===

Two Digi XBee devices were used to communicate between the SJ2-Board controller and the LPC4088 board to control the paddle on the game. The XBee devices were programmed using the XTCU programming application provided by Digi International. The SJ2-Board used one XBee as a transmitted and the LPC4088 used another as a receiver. The two modules communicated using UART frames and a single byte represented any change of movement or button presses sent by the controller.

{| class="wikitable"
|-
! scope="col"| XCTU
! scope="col"| Transmitter
! scope="col"| Receiver
|-
! scope="row"| CH
| C
| C
|-
! scope="row"| ID
| 6969
| 6969
|-
! scope="row"| CE
| Coordinator
| Endpoint
|-
! scope="row"| Baud Rate
| 9600
| 9600
|-
|}
[[File:sj2Board.png|380px|thumb|center|SJ2 Board]]
[[File:CMPE146_F16_MW_XbeeModule.jpg|200px|thumb|center|XBee Device]]
 

Two LED buttons were used to interface with the controller as inputs. They were both set up using GPIO.
[[File:VGA LED Button.jpg|200px|thumb|center|LED Button]]
 
[[File:VGA_Controller_Block_Diagram.PNG|380px|thumb|center|Controller Block Diagram]]
 

==== I2S Peripheral and Stereo Audio Decoder ====

The LPC4088 is equipped with an I2S stereo encoder. This module provides a hardware-enabled solution for sending I2S-encoded audio data packets to stereo decoders, which produce the corresponding audio waveform. For this project, since 16-bit stereo audio playback was chosen, data packets transmitted on this interface were sent in the following format:

[[File:I2S_audio_data_word.png]]

Here, each word of data encodes a single packet of audio in two 16-bit sections: one signal level for the left channel, and another for the right. These 4 bytes are sent at a rate of 44100 Hz, resulting in a total data transmission of 176400 bytes a second. Rather than needing to be software-defined, this transmission is automatically handled by hardware using a 8-word deep FIFO queue that manages streaming out the audio data at a fixed rate, saving a significant amount of CPU cycles.

This encoded audio signal was sent to a dedicated external [https://www.nxp.com/docs/en/data-sheet/UDA1380.pdf stereo decoder], which output to a 3.5 mm audio jack connected to speakers in the VGA monitor.

==== LCD Peripheral ====

The LCD peripheral was configured for 16-bit color palette (5 bits of color data for each color level -- red, green, and blue -- and an unused intensity bit), at a resolution of 800 by 600 pixels, and with an active color palette size of 256 colors. Since the logic for driving an LCD display has the same concepts (horizontal blanking, vertical blanking, etc) and timing information for any given resolution was the same, we were able to directly drive a VGA display with the same signals and timing data (after the digital signal had been converted to analog).

[[File:LCD_%2B_VGA_Video_signal.png]]

The timing register configurations needed to achieve the requested video display was derived from a dedicated website for [http://tinyvga.com/vga-timing/800x600@60Hz VGA signal info].

==== External Memory ====

The game's visual assets and tracker audio file (both after being loaded from an SD card) and the game's video buffer required more memory than what is present natively in the microcontroller. To accommodate this, an external 32MiB memory module was used. This module was interfaced with using the external memory controller peripheral present on the board.

=== Software Design ===
==== Controller ====

There is an internal accelerometer on the SJ2 board that is utilized as the movement controls for the paddle movement. The y-axis value of the magnitude is polled 17 times in 17 milliseconds and is divided by 17 to get an average reading per that time frame. The maximum tilt value that can be read by the controller is set to 700 out of the total 1000 (highest possible value). The current tilt value is divided by 31 to get a total of 32 possible readings to represent 5 bits of magnitude.

The controller sends one byte of data to the console whenever there is any change in magnitude on the accelerometer or button changes. The byte contains all the information needed to control the paddle on the screen. The first two bits on the byte represent 0 if there is no button input and 1 if there is a button input. The third bit represents whether the controller is tilted left or right with 0 being left and 1 being right. The last five bits represent the controller accelerometer's magnitude.

[[File:VGA_UART_Frame.PNG]]

==== Audio ====

Sound design was provided to the project (in the form of music and sound effects) through the use of a [https://https://en.wikipedia.org/wiki/Music_tracker tracker audio library]. Source code was adopted from the [https://github.com/schismtracker/schismtracker Schism Tracker] library and altered with the following modifications:

- All UI logic was stripped from the source code

- All file handling logic was reworked to use the SD card reading functions, as the project does not support POSIX file calls

- The [https://www.libsdl.org/ Simple DirectMedia Layer] hooks were replaced with a FreeRTOS task that periodically called the waveform generation callback function, which was passed to the I2S peripheral hooked up to a stereo decoder.

=== Implementation ===

==== Ball Logic ====

The ball is controlled by the buttons and the controller accelerometer values. When the button is pressed, the ball is initially sent with a random velocity. The velocity is determined by a substate timer and is initially set as one of 6 possible values.

{| class="wikitable"
|-
! scope="col"| Substate Count
! scope="col"| X-Velocity
! scope="col"| Y-Velocity
|-
! scope="row"| 0
| 4
| -2
|-
! scope="row"| 1
| -4
| -2
|-
! scope="row"| 2
| 2
| -4
|-
! scope="row"| 3
| -2
| -4
|-
! scope="row"| 4
| 1
| -5
|-
! scope="row"| 5
| -1
| -5
|-
|}

The ball's velocity changes whenever it detects a collision with either the side of the screen, the bricks on the top of the level, or the paddle at the bottom. If colliding with the paddle, the ball's velocity will take into account the paddle's velocity and add it to its current velocity. This is done by adding the paddle's x-velocity and adjusting the y-velocity by normalizing the velocity vectors to have a magnitude of 5. This means the ball will always be moving with a magnitude of 5 in any direction after it hits the paddle. If the ball collides with anything other than the paddle, then the velocity of the ball is changed with no increase or decrease of magnitude. If the ball collides on the side of a wall or block, the x-velocity will reflect (go from negative to positive or vice-versa) and the y-velocity will remain constant. If the ball collides on the top or bottom of a block, the y-velocity will reflect and the x-velocity will remain constant. If it appears that the ball collides with the side and top or bottom of something, then the ball will check whichever face it will collide with first and then calculate the other face. Lastly, when the ball collides with the bottom of the screen, then the player experiences a "death" and the ball will reset on the paddle. If the ball collides with the top of the screen, the screen will scroll up for the user to play on the next level.

Whenever the ball collides with a brick, the brick will disappear. To accomplish this, there is a 2-D array of unsigned integers that contains the values of the bricks. If the integer is 0, then there is no block. If the integer is any value other than 0, then the block is filled and has collision logic. If the ball approaches a region that might have a block, then there will be a check within that 2-D to see if the block is filled or empty. If the block is filled, there will be the collision logic that changes the vector of the ball and sets the integer value in the 2-D array to 0 to represent that the block has been broken.

==== Paddle Movement ====
The paddle movement is controlled by the accelerometer data being wirelessly transmitted from the SJ-Two board to the LPC4088 board. The accelerometer data will map to 63 possible indexes for the paddle location. Originally, the paddle would jump directly into the corresponding index location depending on the accelerometer input. We later iterated on this process by implementing an algorithm to smoothly transition into these indexes. This was done by calculating a paddle velocity using the "current desired paddle position" and the previous paddle position. We adjusted the parameters of the algorithm iteratively to produce the smooth paddle movement.

==== Paddle Collision Logic ====
The position of the ball is calculated by adding delta_x and delta_y to the ball's current x and y position. For each frame, if the ball's next Y position crosses the paddle's Y position, we would check for a paddle collision. To do this, we used the delta_x and delta_y to get the ball's movement as a slope. We then substituted the slope, the ball's current position, and 1 pixel above the paddle's Y position into a point-slope equation (y2 - y1) = m(x2 - x1) to get the paddle's X position at the time of the ball reaching the pixel above paddle's Y position. After finding the location where the ball is expected to collide with the paddle, we then calculated the ball's collision magnitude and the ball's next location's magnitude. Using a ratio of these two magnitudes, we have a frame time fraction of when the ball possibly collided with the paddle. Then, by using the paddle's current position and paddle velocity, we are able to find where the paddle previously was, and measure how far the paddle had moved during that time frame. By multiplying the time frame from the ball's collision magnitude to the paddle's change in X position, we are able to find whether the paddle was under the ball at the time of the ball reaching the pixel above the paddle's Y position.

== Testing & Technical Challenges ==

=== VGA Interfacing ===
A DAC resistor ladder was designed and a corresponding custom PCB was acquired to interface the 3.3V digital LCD controller signal with the analog VGA input signal. The VGA cable takes between 0V (completely dark) and 0.7V (maximum brightness) for color pin inputs. The DAC resistor ladder was designed and implemented on a breadboard to verify the design. Validating the design on a breadboard gave us a clear vision during PCB design and planning.

=== VGA Write Timings ===
There were initial tests that were required to be done to see if using a VGA monitor as an output was a viable option for this project. We had to test memory write timings to see if they were fast enough to match the refresh rate of the monitor. Those tests would decide how much of the screen buffer we could write to during the blanking periods during the VGA refresh. This was a good learning experience for testing hardware capabilities and should be done whenever introducing any new aspects to a project.

=== Ball Collision Logic ===
The initial version of the ball collision used pixel values to see if the ball was entering the brick. The problem was that this logic did not account for the case where the ball enters the brick diagonally and the logic for that version of the collision had to be scrapped. The later iteration of the logic was changed to checking whether the ball was potentially crossing into an area that had a brick. This was done by separating the checks into a grid that represented the locations of the blocks. In this case, there was an initial check to see if the ball was crossing a boundary where there might be a new brick. If the ball is crossing the x-boundary and not the y-boundary, then there would be a collision check to see if it collided with a block. If it is crossing the y-boundary and not the x-boundary, once again then there would be a collision check to see if it collided with a block. If it is crossing both the x-boundary and y-boundary, then there would be a check to see which boundary the ball would cross first. This new logic allowed for us to account for the case where the block enters the block diagonally. This taught us that sometimes it is beneficial to scrap a previous iteration and use a new approach to solving a problem because it may lead to better results.

=== Paddle Movement ===
One issue we discovered was the paddle would slow down as it approached the walls. This was due to the difference in "current desired paddle position" and the previous paddle position becoming smaller as the walls were approached, leading to a smaller paddle velocity. To account for this, whenever the accelerometer sent data that mapped to the minimum or maximum paddle index, we would increase the distance of the "current desired paddle position" so the paddle velocity would not slow the paddle down as it approached the walls.

=== Paddle Collision Logic ===
During final integration and testing, one of the biggest issues we encountered was a bug where the ball would not detect a collision with the paddle. The ball would phase right through the paddle instead of bouncing off of it. One of the great ideas that we thought of was to change the color of the ball during each check to see if the checks would go through. This debug check allowed us to verify that section of code that checked for paddle collisions was being skipped even though the ball's next Y position had crossed the paddle's Y position. By having this debugging method, we were able to root cause the issue.

== Conclusion ==
For some of us, this was our first experience in designing and implementing a functional embedded system using FreeRTOS. The challenges we hurdled to accomplish our technical goals helped to shape a journey that was greatly enhanced by the opportunity to meet in person and having a hands-on approach to hardware and component integration. We were able to utilize design principles and implement communication protocols we learned in lecture to pull this project together. Some useful topics include: UART, SPI, I2C, I2S, DMA, FreeRTOS mutexes and binary semaphores, queues, task creation, stack size, task scheduler, interrupt service routines, bootloaders, etc. After extensive hours spent testing and verifying our implementation, we have learned that time allocated for integration and testing will significantly improve chances of a successful project. If we had skipped out on testing and verification, the project could have possibly crashed and burned when it was delivered. We hope this project can be of use to any future engineers following behind us, and we wish you all good times and happy development ahead!

=== Project Video ===
[https://www.youtube.com/watch?v=ipngAGlN86c SJSU CMPE 244 FALL 2021: Embedded Software Project Gameplay]

=== Project Source Code ===
* [https://gitlab.com/fayek.wahhab/sjtwo-c/-/tree/feature/i2s_dma2 Treasure Dive Baseline for Demo Source Code]
* [https://gitlab.com/hobriandh/sjtwo-c/-/tree/final_controller Controller for Demo Source Code]

== References ==
=== Acknowledgement ===
We would like to acknowledge and thank our instructor Preetpal Kang for sharing his knowledge in embedded software development using FreeRTOS.
 
Another shoutout to our art designer Jacob for the unique artwork displayed in our game.

=== Appendix ===
*[https://gitlab.com/sjtwo-c-dev/sjtwo-c/-/blob/master/LPC408X_7X%20User%20Manual.pdf LPC408x/407x Datasheet]
*[https://www.freertos.org/Documentation/RTOS_book.html FreeRTOS Documentation]
*[http://elm-chan.org/fsw/ff/00index_e.html FatFs - Generic FAT Filesystem Module]
*[https://www.digi.com/resources/library/data-sheets/ds_xbeemultipointmodules Xbee Datasheet]
*[http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems Embedded Drivers & Real Time Operating Systems Lessons]

F21: Treasure Dive

2021-12-18T12:51:48Z

Proj user1: /* Objectives & Introduction */

== '''Treasure Dive''' ==
[[File:mermaid.png|thumb|500px|right|Game Screen Capture]]
[[File:Cmpe-244-controller-pic.png|thumb|600px|right|Game Controller]]

== Abstract ==
Treasure Dive is a modern one player game inspired by the popular arcade game Breakout, which was released by Atari in 1976. The ancients rambled about an unconfirmed tale of a shipwreck that saw no survivors. As an adventurous and skilled free diver, you are seeking treasure buried deep in an ocean abyss. You dove down all the way to the seafloor and found the buried treasure but need to make it back to the surface to see another day! The player uses a wireless controller to control an on-screen paddle to break bricks on the top of the screen. When enough bricks are broken, a passage is revealed, allowing you to get closer to the surface.

== Objectives & Introduction ==
The objective of this project is to:
 
- interface an LPC4088 to a VGA monitor by leveraging the LCD controller and to use an accelerometer as the sensor for the player controller
 
- integrate two Zigbees RF modules integrated for wireless communication between the controller and LPC4088 board via UART protocol
 
- enable digital audio output by developing an I2S driver and porting an audio tracker library
 
- utilize the General Purpose Direct Memory Access (GPDMA) controller memory-to-memory function to decrease memory write times when updating the video buffer
 
- utilize the DMA controller's memory-to-peripheral function to help reduce frequent I2S interrupts when loading digital audio contents
 
- implement an SD card peripheral driver to facilitate memory exchange between an SD card

=== Team Members & Responsibilities ===
[[File:Cmpe-244-brian-pic.png ‎]]
* Brian Ho
** Wiki project schedule planning
** Interfacing the accelerometer
** Digital to Analog Conversion resistor ladder design used to interface digital LCD controller signal with analog VGA port signal
** PCB planning and acquisition
** Procurement of controller hardware
** Wireless communication via Zigbees RF modules and UART protocol
** Wireless controller logic and integration
** Ball movement and wall/brick collision logic
** Hardware/software development and integration
** Final testing and integration
[[File:Cmpe-244-billy-pic.png]]
* [https://www.linkedin.com/in/william-lai-525681139 Billy Lai]
** Wiki project schedule planning
** Increased CPU clock rate from 96 MHz to 120 MHz
** Initial memory write timing and measurements to estimate memory write capacity
** DMA driver memory-to-memory implementation for transferring graphics data to video buffer
** I2S with DMA memory-to-peripheral implementation to reduce memory-write interrupts when I2S Tx FIFO level is low
** Paddle movement and paddle collision logic, testing and integration
** Software development and integration
** Final testing and integration
[[File:Cmpe-244-jasdip-pic.png]]
* Jasdip Sekhon
** Wiki project schedule planning
** PCB design and planning
** File loading method algorithm for storing loaded files in memory
** Audio track and audio effects selection for gameplay audio
** Palette manipulation for gameplay
** Software development and integration
** Final testing and integration
[[File:Cmpe-244-isaac-pic.png ]]
* Isaac Wahhab
** Wiki project schedule planning
** DAC resistor ladder design and integration
** Hardware procurement (LPC4088 board, Zigbees, VGA monitor, etc)
** VGA driving through LCD controller
** Audio tracker library port
** Digital audio output using I2S
** Palette scrolling feature implementation
** Graphics rendering
** Vertical scrolling implementation for the video buffer
** Sprite testing and implementation for the ball
** SD card peripheral driver for reading assets from SD card
** Art assets acquisition
** Hardware/software development and integration
** Gameplay logic design
** Gameplay audio and level design
** Final testing and integration

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
*09/21/2021
|
*09/27/2021
|
* Read previous projects, gather information and discuss among the group members.
* Initial ordering of parts (LPC 4088 board, 64x64 LED screen)
|
* Completed
* Completed
|-
! scope="row"| 2
|
*09/28/2021
|
*10/04/2021
|
* Submit project proposals
|
* Completed
|-
! scope="row"| 3
|
*10/05/2021
|
*10/11/2021
|
* Test if board can drive VGA
* Test varying clock rates
* Hardware cursor on VGA
* Rough display driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 4
|
* 10/12/2021
|
* 10/18/2021
|
* DAC resistor ladder
* RGB channels
* Palette shifting
* Measure memory buffer writing timing
* Wiki schedule planning
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
*10/19/2021
|
*10/25/2021
|
* Calculate upper-bound for VGA refresh timings
* Create GitLab repository for project
* Joystick functionality
* SD card driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
*10/26/2021
|
*11/01/2021
|
* Audio peripheral
* Wireless communication using Zigbee module
* PCB planning
* Accelerometer driver
* Continue SD card driver
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/02/2021

|
* 11/08/2021
|
* Continue audio peripheral
* Continue wireless communication using Zigbee module
* Continue accelerometer driver
* PCB finalizing and ordering
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 11/09/2021
|
* 11/15/2021
|
* Convert PNG data to graphics format
* Game background functionality
* Sprite functionality
* User interface
* Integrate components
* Initial testing
|
* Completed
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 11/16/2021
|
* 11/22/2021
|
* Continue everything from last week
* Game physics (wall collision and AI movement)
* Measurements for wooden case
* Art assets (audio and visual)
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 10
|
*11/23/2021
|
*11/29/2021
|

* Fine-tune user experience
* Make case
|
* Completed
* Completed
|-
! scope="row"| 11
|
* 11/30/2021
|
* 12/06/2021
|
* Continue fine-tune user experience
* Integrate hardware
|
* Completed
* Completed
|-
! scope="row"| 12
|
* 12/07/2021
|
* 12/16/2021
|
* Final testing
* Final demo
|
* Completed
* Completed
|-
|}

== Bill Of Materials ==
{| class="wikitable" style="margin-left: 0px; margin-right: auto;"
|-
! scope="col"| Item#
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| [https://www.embeddedartists.com/products/lpc4088-developers-kit LPC4088-32 Developer's Kit]
| 1
| $300.00
|-
! scope="row"| 2
| Microcontroller Board
| [http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems/page/sj2-board SJ2 Board]
| 1
| $50.00
|-
! scope="row"| 3
| Bluetooth Module
| [https://hub.digi.com/support/products/digi-xbee/digi-xbee-s1 Digi XBee-S1]
| 2
| N/A
|-
! scope="row"| 4
| DAC PCB
| [https://jlcpcb.com JLCPCB (Set of 5)]
| 1
| $15.00
|-
! scope="row"| 4
| Monitor
| Asus TFT Monitor
| 1
| N/A
|-
! scope="row"| 5
| Mini Breadboard
| [https://www.amazon.com/EL-CK-002-Electronic-Breadboard-Capacitor-Potentiometer/dp/B01ERP6WL4/ref=pd_sbs_147_2/132-6936347-2117569?_encoding=UTF8&pd_rd_i=B01ERP6WL4&pd_rd_r=6a783095-ec87-42ae-9f01-a5ebc9a44607&pd_rd_w=V2aU6&pd_rd_wg=11h1i&pf_rd_p=d66372fe-68a6-48a3-90ec-41d7f64212be&pf_rd_r=368WNKXFBBQY2FJQ4119&psc=1&refRID=368WNKXFBBQY2FJQ4119 ELEGOO Mini Breadboard]
| 1
| $13.99
|-
! scope="row"| 6
| Arcade buttons
| [https://www.amazon.com/EG-STARTS-Illuminated-Buttons-Raspberry/dp/B01N11BDX9?pd_rd_w=DiSpt&pf_rd_p=5bc65336-a8b0-4888-9e70-4d4ce2f6bd59&pf_rd_r=JCH2VQYXFPHND4XZ4FAM&pd_rd_r=8d6243dd-e1bd-4dae-9b19-be2e00f3085f&pd_rd_wg=K4lt9&pd_rd_i=B01N11BDX9&psc=1&ref_=pd_bap_d_rp_1_i EG STARTS LED Arcade Buttons]
| 2
| $11.99
|-
! scope="row"| 7
| VGA cable
| N/A
| 1
| N/A
|-
! scope="row"| 7
| SD card
| N/A
| 1
| N/A
|-
|}

== Design & Implementation ==

=== Hardware Design ===
We designed our PCB using Autodesk EAGLE based on the tutorial on PCB design during the class lecture. We ordered our PCB to be manufactured from JLCPCB. The PCB consists of an R-2R resistor ladder DAC for the purpose of converting the digital signal from the microcontroller to analog to display graphics on the VGA monitor. Since the VGA input for each color channel is 0-0.7V, the resistor ladder scaled the LPC4088 outputs into proper DAC values for the VGA to take in. Five bits are passed to each of the red, green, and blue channels of the VGA in addition to other signals, such as Horizontal sync (HSYNC) and Vertical sync (VSYNC).
 
[[File:pcbSchematic.png|500px|thumb|left|PCB Schematic]]
[[File:pcbLayout.png|500px|thumb|right|PCB Layout]]
[[File:pcbPic-min.png|500px|thumb|right|Manufactured PCB]]
[[File:pcbManufactured.png|500px|thumb|left|Soldered PCB]]
 

=== Hardware Interface ===

Two Digi XBee devices were used to communicate between the SJ2-Board controller and the LPC4088 board to control the paddle on the game. The XBee devices were programmed using the XTCU programming application provided by Digi International. The SJ2-Board used one XBee as a transmitted and the LPC4088 used another as a receiver. The two modules communicated using UART frames and a single byte represented any change of movement or button presses sent by the controller.

{| class="wikitable"
|-
! scope="col"| XCTU
! scope="col"| Transmitter
! scope="col"| Receiver
|-
! scope="row"| CH
| C
| C
|-
! scope="row"| ID
| 6969
| 6969
|-
! scope="row"| CE
| Coordinator
| Endpoint
|-
! scope="row"| Baud Rate
| 9600
| 9600
|-
|}
[[File:sj2Board.png|380px|thumb|center|SJ2 Board]]
[[File:CMPE146_F16_MW_XbeeModule.jpg|200px|thumb|center|XBee Device]]
 

Two LED buttons were used to interface with the controller as inputs. They were both set up using GPIO.
[[File:VGA LED Button.jpg|200px|thumb|center|LED Button]]
 
[[File:VGA_Controller_Block_Diagram.PNG|380px|thumb|center|Controller Block Diagram]]
 

==== I2S Peripheral and Stereo Audio Decoder ====

The LPC4088 is equipped with an I2S stereo encoder. This module provides a hardware-enabled solution for sending I2S-encoded audio data packets to stereo decoders, which produce the corresponding audio waveform. For this project, since 16-bit stereo audio playback was chosen, data packets transmitted on this interface were sent in the following format:

[[File:I2S_audio_data_word.png]]

Here, each word of data encodes a single packet of audio in two 16-bit sections: one signal level for the left channel, and another for the right. These 4 bytes are sent at a rate of 44100 Hz, resulting in a total data transmission of 176400 bytes a second. Rather than needing to be software-defined, this transmission is automatically handled by hardware using a 8-word deep FIFO queue that manages streaming out the audio data at a fixed rate, saving a significant amount of CPU cycles.

This encoded audio signal was sent to a dedicated external [https://www.nxp.com/docs/en/data-sheet/UDA1380.pdf stereo decoder], which output to a 3.5 mm audio jack connected to speakers in the VGA monitor.

==== LCD Peripheral ====

The LCD peripheral was configured for 16-bit color palette (5 bits of color data for each color level -- red, green, and blue -- and an unused intensity bit), at a resolution of 800 by 600 pixels, and with an active color palette size of 256 colors. Since the logic for driving an LCD display has the same concepts (horizontal blanking, vertical blanking, etc) and timing information for any given resolution was the same, we were able to directly drive a VGA display with the same signals and timing data (after the digital signal had been converted to analog).

[[File:LCD_%2B_VGA_Video_signal.png]]

The timing register configurations needed to achieve the requested video display was derived from a dedicated website for [http://tinyvga.com/vga-timing/800x600@60Hz VGA signal info].

==== External Memory ====

The game's visual assets and tracker audio file (both after being loaded from an SD card) and the game's video buffer required more memory than what is present natively in the microcontroller. To accommodate this, an external 32MiB memory module was used. This module was interfaced with using the external memory controller peripheral present on the board.

=== Software Design ===
==== Controller ====

There is an internal accelerometer on the SJ2 board that is utilized as the movement controls for the paddle movement. The y-axis value of the magnitude is polled 17 times in 17 milliseconds and is divided by 17 to get an average reading per that time frame. The maximum tilt value that can be read by the controller is set to 700 out of the total 1000 (highest possible value). The current tilt value is divided by 31 to get a total of 32 possible readings to represent 5 bits of magnitude.

The controller sends one byte of data to the console whenever there is any change in magnitude on the accelerometer or button changes. The byte contains all the information needed to control the paddle on the screen. The first two bits on the byte represent 0 if there is no button input and 1 if there is a button input. The third bit represents whether the controller is tilted left or right with 0 being left and 1 being right. The last five bits represent the controller accelerometer's magnitude.

[[File:VGA_UART_Frame.PNG]]

==== Audio ====

Sound design was provided to the project (in the form of music and sound effects) through the use of a [https://https://en.wikipedia.org/wiki/Music_tracker tracker audio library]. Source code was adopted from the [https://github.com/schismtracker/schismtracker Schism Tracker] library and altered with the following modifications:

- All UI logic was stripped from the source code

- All file handling logic was reworked to use the SD card reading functions, as the project does not support POSIX file calls

- The [https://www.libsdl.org/ Simple DirectMedia Layer] hooks were replaced with a FreeRTOS task that periodically called the waveform generation callback function, which was passed to the I2S peripheral hooked up to a stereo decoder.

=== Implementation ===

==== Ball Logic ====

The ball is controlled by the buttons and the controller accelerometer values. When the button is pressed, the ball is initially sent with a random velocity. The velocity is determined by a substate timer and is initially set as one of 6 possible values.

{| class="wikitable"
|-
! scope="col"| Substate Count
! scope="col"| X-Velocity
! scope="col"| Y-Velocity
|-
! scope="row"| 0
| 4
| -2
|-
! scope="row"| 1
| -4
| -2
|-
! scope="row"| 2
| 2
| -4
|-
! scope="row"| 3
| -2
| -4
|-
! scope="row"| 4
| 1
| -5
|-
! scope="row"| 5
| -1
| -5
|-
|}

The ball's velocity changes whenever it detects a collision with either the side of the screen, the bricks on the top of the level, or the paddle at the bottom. If colliding with the paddle, the ball's velocity will take into account the paddle's velocity and add it to its current velocity. This is done by adding the paddle's x-velocity and adjusting the y-velocity by normalizing the velocity vectors to have a magnitude of 5. This means the ball will always be moving with a magnitude of 5 in any direction after it hits the paddle. If the ball collides with anything other than the paddle, then the velocity of the ball is changed with no increase or decrease of magnitude. If the ball collides on the side of a wall or block, the x-velocity will reflect (go from negative to positive or vice-versa) and the y-velocity will remain constant. If the ball collides on the top or bottom of a block, the y-velocity will reflect and the x-velocity will remain constant. If it appears that the ball collides with the side and top or bottom of something, then the ball will check whichever face it will collide with first and then calculate the other face. Lastly, when the ball collides with the bottom of the screen, then the player experiences a "death" and the ball will reset on the paddle. If the ball collides with the top of the screen, the screen will scroll up for the user to play on the next level.

Whenever the ball collides with a brick, the brick will disappear. To accomplish this, there is a 2-D array of unsigned integers that contains the values of the bricks. If the integer is 0, then there is no block. If the integer is any value other than 0, then the block is filled and has collision logic. If the ball approaches a region that might have a block, then there will be a check within that 2-D to see if the block is filled or empty. If the block is filled, there will be the collision logic that changes the vector of the ball and sets the integer value in the 2-D array to 0 to represent that the block has been broken.

==== Paddle Movement ====
The paddle movement is controlled by the accelerometer data being wirelessly transmitted from the SJ-Two board to the LPC4088 board. The accelerometer data will map to 63 possible indexes for the paddle location. Originally, the paddle would jump directly into the corresponding index location depending on the accelerometer input. We later iterated on this process by implementing an algorithm to smoothly transition into these indexes. This was done by calculating a paddle velocity using the "current desired paddle position" and the previous paddle position. We adjusted the parameters of the algorithm iteratively to produce the smooth paddle movement.

==== Paddle Collision Logic ====
The position of the ball is calculated by adding delta_x and delta_y to the ball's current x and y position. For each frame, if the ball's next Y position crosses the paddle's Y position, we would check for a paddle collision. To do this, we used the delta_x and delta_y to get the ball's movement as a slope. We then substituted the slope, the ball's current position, and 1 pixel above the paddle's Y position into a point-slope equation (y2 - y1) = m(x2 - x1) to get the paddle's X position at the time of the ball reaching the pixel above paddle's Y position. After finding the location where the ball is expected to collide with the paddle, we then calculated the ball's collision magnitude and the ball's next location's magnitude. Using a ratio of these two magnitudes, we have a frame time fraction of when the ball possibly collided with the paddle. Then, by using the paddle's current position and paddle velocity, we are able to find where the paddle previously was, and measure how far the paddle had moved during that time frame. By multiplying the time frame from the ball's collision magnitude to the paddle's change in X position, we are able to find whether the paddle was under the ball at the time of the ball reaching the pixel above the paddle's Y position.

== Testing & Technical Challenges ==

=== VGA Interfacing ===
A DAC resistor ladder was designed and a corresponding custom PCB was acquired to interface the 3.3V digital LCD controller signal with the analog VGA input signal. The VGA cable takes between 0V (completely dark) and 0.7V (maximum brightness) for color pin inputs. The DAC resistor ladder was designed and implemented on a breadboard to verify the design. Validating the design on a breadboard gave us a clear vision during PCB design and planning.

=== VGA Write Timings ===
There were initial tests that were required to be done to see if using a VGA monitor as an output was a viable option for this project. We had to test memory write timings to see if they were fast enough to match the refresh rate of the monitor. Those tests would decide how much of the screen buffer we could write to during the blanking periods during the VGA refresh. This was a good learning experience for testing hardware capabilities and should be done whenever introducing any new aspects to a project.

=== Ball Collision Logic ===
The initial version of the ball collision used pixel values to see if the ball was entering the brick. The problem was that this logic did not account for the case where the ball enters the brick diagonally and the logic for that version of the collision had to be scrapped. The later iteration of the logic was changed to checking whether the ball was potentially crossing into an area that had a brick. This was done by separating the checks into a grid that represented the locations of the blocks. In this case, there was an initial check to see if the ball was crossing a boundary where there might be a new brick. If the ball is crossing the x-boundary and not the y-boundary, then there would be a collision check to see if it collided with a block. If it is crossing the y-boundary and not the x-boundary, once again then there would be a collision check to see if it collided with a block. If it is crossing both the x-boundary and y-boundary, then there would be a check to see which boundary the ball would cross first. This new logic allowed for us to account for the case where the block enters the block diagonally. This taught us that sometimes it is beneficial to scrap a previous iteration and use a new approach to solving a problem because it may lead to better results.

=== Paddle Movement ===
One issue we discovered was the paddle would slow down as it approached the walls. This was due to the difference in "current desired paddle position" and the previous paddle position becoming smaller as the walls were approached, leading to a smaller paddle velocity. To account for this, whenever the accelerometer sent data that mapped to the minimum or maximum paddle index, we would increase the distance of the "current desired paddle position" so the paddle velocity would not slow the paddle down as it approached the walls.

=== Paddle Collision Logic ===
During final integration and testing, one of the biggest issues we encountered was a bug where the ball would not detect a collision with the paddle. The ball would phase right through the paddle instead of bouncing off of it. One of the great ideas that we thought of was to change the color of the ball during each check to see if the checks would go through. This debug check allowed us to verify that section of code that checked for paddle collisions was being skipped even though the ball's next Y position had crossed the paddle's Y position. By having this debugging method, we were able to root cause the issue.

== Conclusion ==
For some of us, this was our first experience in designing and implementing a functional embedded system using FreeRTOS. The challenges we hurdled to accomplish our technical goals helped to shape a journey that was greatly enhanced by the opportunity to meet in person and having a hands-on approach to hardware and component integration. We were able to utilize design principles and implement communication protocols we learned in lecture to pull this project together. Some useful topics include: UART, SPI, I2C, I2S, DMA, FreeRTOS mutexes and binary semaphores, queues, task creation, stack size, task scheduler, interrupt service routines, bootloaders, etc. After extensive hours spent testing and verifying our implementation, we have learned that time allocated for integration and testing will significantly improve chances of a successful project. If we had skipped out on testing and verification, the project could have possibly crashed and burned when it was delivered. We hope this project can be of use to any future engineers following behind us, and we wish you all good times and happy development ahead!

=== Project Video ===
[https://www.youtube.com/watch?v=ipngAGlN86c SJSU CMPE 244 FALL 2021: Embedded Software Project Gameplay]

=== Project Source Code ===
* [https://gitlab.com/fayek.wahhab/sjtwo-c/-/tree/feature/i2s_dma2 Treasure Dive Baseline for Demo Source Code]
* [https://gitlab.com/hobriandh/sjtwo-c/-/tree/final_controller Controller for Demo Source Code]

== References ==
=== Acknowledgement ===
We would like to acknowledge and thank our instructor Preetpal Kang for sharing his knowledge in embedded software development using FreeRTOS.
 
Another shoutout to our art designer Jacob for the unique artwork displayed in our game.

=== Appendix ===
*[https://gitlab.com/sjtwo-c-dev/sjtwo-c/-/blob/master/LPC408X_7X%20User%20Manual.pdf LPC408x/407x Datasheet]
*[https://www.freertos.org/Documentation/RTOS_book.html FreeRTOS Documentation]
*[http://elm-chan.org/fsw/ff/00index_e.html FatFs - Generic FAT Filesystem Module]
*[https://www.digi.com/resources/library/data-sheets/ds_xbeemultipointmodules Xbee Datasheet]
*[http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems Embedded Drivers & Real Time Operating Systems Lessons]

F21: Treasure Dive

2021-12-18T12:49:28Z

Proj user1: /* Objectives & Introduction */

== '''Treasure Dive''' ==
[[File:mermaid.png|thumb|500px|right|Game Screen Capture]]
[[File:Cmpe-244-controller-pic.png|thumb|600px|right|Game Controller]]

== Abstract ==
Treasure Dive is a modern one player game inspired by the popular arcade game Breakout, which was released by Atari in 1976. The ancients rambled about an unconfirmed tale of a shipwreck that saw no survivors. As an adventurous and skilled free diver, you are seeking treasure buried deep in an ocean abyss. You dove down all the way to the seafloor and found the buried treasure but need to make it back to the surface to see another day! The player uses a wireless controller to control an on-screen paddle to break bricks on the top of the screen. When enough bricks are broken, a passage is revealed, allowing you to get closer to the surface.

== Objectives & Introduction ==
The objective of this project is to interface an LPC4088 to a VGA monitor by leveraging the LCD controller and to use an accelerometer as the sensor for the player controller.
 
Two Zigbees RF modules will be integrated for wireless communication between the controller and LPC4088 board via UART protocol.
 
Digital audio output will be enabled by developing an I2S driver and porting an audio tracker library.
 
The General Purpose Direct Memory Access (GPDMA) controller memory-to-memory function will be utilized to decrease memory write times when updating the video buffer.
 
The DMA controller's memory-to-peripheral function will help reduce the frequent I2S interrupts when loading digital audio contents.
 
An SD card peripheral driver to facilitate memory exchange between an SD card will also be implemented.

=== Team Members & Responsibilities ===
[[File:Cmpe-244-brian-pic.png ‎]]
* Brian Ho
** Wiki project schedule planning
** Interfacing the accelerometer
** Digital to Analog Conversion resistor ladder design used to interface digital LCD controller signal with analog VGA port signal
** PCB planning and acquisition
** Procurement of controller hardware
** Wireless communication via Zigbees RF modules and UART protocol
** Wireless controller logic and integration
** Ball movement and wall/brick collision logic
** Hardware/software development and integration
** Final testing and integration
[[File:Cmpe-244-billy-pic.png]]
* [https://www.linkedin.com/in/william-lai-525681139 Billy Lai]
** Wiki project schedule planning
** Increased CPU clock rate from 96 MHz to 120 MHz
** Initial memory write timing and measurements to estimate memory write capacity
** DMA driver memory-to-memory implementation for transferring graphics data to video buffer
** I2S with DMA memory-to-peripheral implementation to reduce memory-write interrupts when I2S Tx FIFO level is low
** Paddle movement and paddle collision logic, testing and integration
** Software development and integration
** Final testing and integration
[[File:Cmpe-244-jasdip-pic.png]]
* Jasdip Sekhon
** Wiki project schedule planning
** PCB design and planning
** File loading method algorithm for storing loaded files in memory
** Audio track and audio effects selection for gameplay audio
** Palette manipulation for gameplay
** Software development and integration
** Final testing and integration
[[File:Cmpe-244-isaac-pic.png ]]
* Isaac Wahhab
** Wiki project schedule planning
** DAC resistor ladder design and integration
** Hardware procurement (LPC4088 board, Zigbees, VGA monitor, etc)
** VGA driving through LCD controller
** Audio tracker library port
** Digital audio output using I2S
** Palette scrolling feature implementation
** Graphics rendering
** Vertical scrolling implementation for the video buffer
** Sprite testing and implementation for the ball
** SD card peripheral driver for reading assets from SD card
** Art assets acquisition
** Hardware/software development and integration
** Gameplay logic design
** Gameplay audio and level design
** Final testing and integration

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
*09/21/2021
|
*09/27/2021
|
* Read previous projects, gather information and discuss among the group members.
* Initial ordering of parts (LPC 4088 board, 64x64 LED screen)
|
* Completed
* Completed
|-
! scope="row"| 2
|
*09/28/2021
|
*10/04/2021
|
* Submit project proposals
|
* Completed
|-
! scope="row"| 3
|
*10/05/2021
|
*10/11/2021
|
* Test if board can drive VGA
* Test varying clock rates
* Hardware cursor on VGA
* Rough display driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 4
|
* 10/12/2021
|
* 10/18/2021
|
* DAC resistor ladder
* RGB channels
* Palette shifting
* Measure memory buffer writing timing
* Wiki schedule planning
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
*10/19/2021
|
*10/25/2021
|
* Calculate upper-bound for VGA refresh timings
* Create GitLab repository for project
* Joystick functionality
* SD card driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
*10/26/2021
|
*11/01/2021
|
* Audio peripheral
* Wireless communication using Zigbee module
* PCB planning
* Accelerometer driver
* Continue SD card driver
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/02/2021

|
* 11/08/2021
|
* Continue audio peripheral
* Continue wireless communication using Zigbee module
* Continue accelerometer driver
* PCB finalizing and ordering
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 11/09/2021
|
* 11/15/2021
|
* Convert PNG data to graphics format
* Game background functionality
* Sprite functionality
* User interface
* Integrate components
* Initial testing
|
* Completed
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 11/16/2021
|
* 11/22/2021
|
* Continue everything from last week
* Game physics (wall collision and AI movement)
* Measurements for wooden case
* Art assets (audio and visual)
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 10
|
*11/23/2021
|
*11/29/2021
|

* Fine-tune user experience
* Make case
|
* Completed
* Completed
|-
! scope="row"| 11
|
* 11/30/2021
|
* 12/06/2021
|
* Continue fine-tune user experience
* Integrate hardware
|
* Completed
* Completed
|-
! scope="row"| 12
|
* 12/07/2021
|
* 12/16/2021
|
* Final testing
* Final demo
|
* Completed
* Completed
|-
|}

== Bill Of Materials ==
{| class="wikitable" style="margin-left: 0px; margin-right: auto;"
|-
! scope="col"| Item#
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| [https://www.embeddedartists.com/products/lpc4088-developers-kit LPC4088-32 Developer's Kit]
| 1
| $300.00
|-
! scope="row"| 2
| Microcontroller Board
| [http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems/page/sj2-board SJ2 Board]
| 1
| $50.00
|-
! scope="row"| 3
| Bluetooth Module
| [https://hub.digi.com/support/products/digi-xbee/digi-xbee-s1 Digi XBee-S1]
| 2
| N/A
|-
! scope="row"| 4
| DAC PCB
| [https://jlcpcb.com JLCPCB (Set of 5)]
| 1
| $15.00
|-
! scope="row"| 4
| Monitor
| Asus TFT Monitor
| 1
| N/A
|-
! scope="row"| 5
| Mini Breadboard
| [https://www.amazon.com/EL-CK-002-Electronic-Breadboard-Capacitor-Potentiometer/dp/B01ERP6WL4/ref=pd_sbs_147_2/132-6936347-2117569?_encoding=UTF8&pd_rd_i=B01ERP6WL4&pd_rd_r=6a783095-ec87-42ae-9f01-a5ebc9a44607&pd_rd_w=V2aU6&pd_rd_wg=11h1i&pf_rd_p=d66372fe-68a6-48a3-90ec-41d7f64212be&pf_rd_r=368WNKXFBBQY2FJQ4119&psc=1&refRID=368WNKXFBBQY2FJQ4119 ELEGOO Mini Breadboard]
| 1
| $13.99
|-
! scope="row"| 6
| Arcade buttons
| [https://www.amazon.com/EG-STARTS-Illuminated-Buttons-Raspberry/dp/B01N11BDX9?pd_rd_w=DiSpt&pf_rd_p=5bc65336-a8b0-4888-9e70-4d4ce2f6bd59&pf_rd_r=JCH2VQYXFPHND4XZ4FAM&pd_rd_r=8d6243dd-e1bd-4dae-9b19-be2e00f3085f&pd_rd_wg=K4lt9&pd_rd_i=B01N11BDX9&psc=1&ref_=pd_bap_d_rp_1_i EG STARTS LED Arcade Buttons]
| 2
| $11.99
|-
! scope="row"| 7
| VGA cable
| N/A
| 1
| N/A
|-
! scope="row"| 7
| SD card
| N/A
| 1
| N/A
|-
|}

== Design & Implementation ==

=== Hardware Design ===
We designed our PCB using Autodesk EAGLE based on the tutorial on PCB design during the class lecture. We ordered our PCB to be manufactured from JLCPCB. The PCB consists of an R-2R resistor ladder DAC for the purpose of converting the digital signal from the microcontroller to analog to display graphics on the VGA monitor. Since the VGA input for each color channel is 0-0.7V, the resistor ladder scaled the LPC4088 outputs into proper DAC values for the VGA to take in. Five bits are passed to each of the red, green, and blue channels of the VGA in addition to other signals, such as Horizontal sync (HSYNC) and Vertical sync (VSYNC).
 
[[File:pcbSchematic.png|500px|thumb|left|PCB Schematic]]
[[File:pcbLayout.png|500px|thumb|right|PCB Layout]]
[[File:pcbPic-min.png|500px|thumb|right|Manufactured PCB]]
[[File:pcbManufactured.png|500px|thumb|left|Soldered PCB]]
 

=== Hardware Interface ===

Two Digi XBee devices were used to communicate between the SJ2-Board controller and the LPC4088 board to control the paddle on the game. The XBee devices were programmed using the XTCU programming application provided by Digi International. The SJ2-Board used one XBee as a transmitted and the LPC4088 used another as a receiver. The two modules communicated using UART frames and a single byte represented any change of movement or button presses sent by the controller.

{| class="wikitable"
|-
! scope="col"| XCTU
! scope="col"| Transmitter
! scope="col"| Receiver
|-
! scope="row"| CH
| C
| C
|-
! scope="row"| ID
| 6969
| 6969
|-
! scope="row"| CE
| Coordinator
| Endpoint
|-
! scope="row"| Baud Rate
| 9600
| 9600
|-
|}
[[File:sj2Board.png|380px|thumb|center|SJ2 Board]]
[[File:CMPE146_F16_MW_XbeeModule.jpg|200px|thumb|center|XBee Device]]
 

Two LED buttons were used to interface with the controller as inputs. They were both set up using GPIO.
[[File:VGA LED Button.jpg|200px|thumb|center|LED Button]]
 
[[File:VGA_Controller_Block_Diagram.PNG|380px|thumb|center|Controller Block Diagram]]
 

==== I2S Peripheral and Stereo Audio Decoder ====

The LPC4088 is equipped with an I2S stereo encoder. This module provides a hardware-enabled solution for sending I2S-encoded audio data packets to stereo decoders, which produce the corresponding audio waveform. For this project, since 16-bit stereo audio playback was chosen, data packets transmitted on this interface were sent in the following format:

[[File:I2S_audio_data_word.png]]

Here, each word of data encodes a single packet of audio in two 16-bit sections: one signal level for the left channel, and another for the right. These 4 bytes are sent at a rate of 44100 Hz, resulting in a total data transmission of 176400 bytes a second. Rather than needing to be software-defined, this transmission is automatically handled by hardware using a 8-word deep FIFO queue that manages streaming out the audio data at a fixed rate, saving a significant amount of CPU cycles.

This encoded audio signal was sent to a dedicated external [https://www.nxp.com/docs/en/data-sheet/UDA1380.pdf stereo decoder], which output to a 3.5 mm audio jack connected to speakers in the VGA monitor.

==== LCD Peripheral ====

The LCD peripheral was configured for 16-bit color palette (5 bits of color data for each color level -- red, green, and blue -- and an unused intensity bit), at a resolution of 800 by 600 pixels, and with an active color palette size of 256 colors. Since the logic for driving an LCD display has the same concepts (horizontal blanking, vertical blanking, etc) and timing information for any given resolution was the same, we were able to directly drive a VGA display with the same signals and timing data (after the digital signal had been converted to analog).

[[File:LCD_%2B_VGA_Video_signal.png]]

The timing register configurations needed to achieve the requested video display was derived from a dedicated website for [http://tinyvga.com/vga-timing/800x600@60Hz VGA signal info].

==== External Memory ====

The game's visual assets and tracker audio file (both after being loaded from an SD card) and the game's video buffer required more memory than what is present natively in the microcontroller. To accommodate this, an external 32MiB memory module was used. This module was interfaced with using the external memory controller peripheral present on the board.

=== Software Design ===
==== Controller ====

There is an internal accelerometer on the SJ2 board that is utilized as the movement controls for the paddle movement. The y-axis value of the magnitude is polled 17 times in 17 milliseconds and is divided by 17 to get an average reading per that time frame. The maximum tilt value that can be read by the controller is set to 700 out of the total 1000 (highest possible value). The current tilt value is divided by 31 to get a total of 32 possible readings to represent 5 bits of magnitude.

The controller sends one byte of data to the console whenever there is any change in magnitude on the accelerometer or button changes. The byte contains all the information needed to control the paddle on the screen. The first two bits on the byte represent 0 if there is no button input and 1 if there is a button input. The third bit represents whether the controller is tilted left or right with 0 being left and 1 being right. The last five bits represent the controller accelerometer's magnitude.

[[File:VGA_UART_Frame.PNG]]

==== Audio ====

Sound design was provided to the project (in the form of music and sound effects) through the use of a [https://https://en.wikipedia.org/wiki/Music_tracker tracker audio library]. Source code was adopted from the [https://github.com/schismtracker/schismtracker Schism Tracker] library and altered with the following modifications:

- All UI logic was stripped from the source code

- All file handling logic was reworked to use the SD card reading functions, as the project does not support POSIX file calls

- The [https://www.libsdl.org/ Simple DirectMedia Layer] hooks were replaced with a FreeRTOS task that periodically called the waveform generation callback function, which was passed to the I2S peripheral hooked up to a stereo decoder.

=== Implementation ===

==== Ball Logic ====

The ball is controlled by the buttons and the controller accelerometer values. When the button is pressed, the ball is initially sent with a random velocity. The velocity is determined by a substate timer and is initially set as one of 6 possible values.

{| class="wikitable"
|-
! scope="col"| Substate Count
! scope="col"| X-Velocity
! scope="col"| Y-Velocity
|-
! scope="row"| 0
| 4
| -2
|-
! scope="row"| 1
| -4
| -2
|-
! scope="row"| 2
| 2
| -4
|-
! scope="row"| 3
| -2
| -4
|-
! scope="row"| 4
| 1
| -5
|-
! scope="row"| 5
| -1
| -5
|-
|}

The ball's velocity changes whenever it detects a collision with either the side of the screen, the bricks on the top of the level, or the paddle at the bottom. If colliding with the paddle, the ball's velocity will take into account the paddle's velocity and add it to its current velocity. This is done by adding the paddle's x-velocity and adjusting the y-velocity by normalizing the velocity vectors to have a magnitude of 5. This means the ball will always be moving with a magnitude of 5 in any direction after it hits the paddle. If the ball collides with anything other than the paddle, then the velocity of the ball is changed with no increase or decrease of magnitude. If the ball collides on the side of a wall or block, the x-velocity will reflect (go from negative to positive or vice-versa) and the y-velocity will remain constant. If the ball collides on the top or bottom of a block, the y-velocity will reflect and the x-velocity will remain constant. If it appears that the ball collides with the side and top or bottom of something, then the ball will check whichever face it will collide with first and then calculate the other face. Lastly, when the ball collides with the bottom of the screen, then the player experiences a "death" and the ball will reset on the paddle. If the ball collides with the top of the screen, the screen will scroll up for the user to play on the next level.

Whenever the ball collides with a brick, the brick will disappear. To accomplish this, there is a 2-D array of unsigned integers that contains the values of the bricks. If the integer is 0, then there is no block. If the integer is any value other than 0, then the block is filled and has collision logic. If the ball approaches a region that might have a block, then there will be a check within that 2-D to see if the block is filled or empty. If the block is filled, there will be the collision logic that changes the vector of the ball and sets the integer value in the 2-D array to 0 to represent that the block has been broken.

==== Paddle Movement ====
The paddle movement is controlled by the accelerometer data being wirelessly transmitted from the SJ-Two board to the LPC4088 board. The accelerometer data will map to 63 possible indexes for the paddle location. Originally, the paddle would jump directly into the corresponding index location depending on the accelerometer input. We later iterated on this process by implementing an algorithm to smoothly transition into these indexes. This was done by calculating a paddle velocity using the "current desired paddle position" and the previous paddle position. We adjusted the parameters of the algorithm iteratively to produce the smooth paddle movement.

==== Paddle Collision Logic ====
The position of the ball is calculated by adding delta_x and delta_y to the ball's current x and y position. For each frame, if the ball's next Y position crosses the paddle's Y position, we would check for a paddle collision. To do this, we used the delta_x and delta_y to get the ball's movement as a slope. We then substituted the slope, the ball's current position, and 1 pixel above the paddle's Y position into a point-slope equation (y2 - y1) = m(x2 - x1) to get the paddle's X position at the time of the ball reaching the pixel above paddle's Y position. After finding the location where the ball is expected to collide with the paddle, we then calculated the ball's collision magnitude and the ball's next location's magnitude. Using a ratio of these two magnitudes, we have a frame time fraction of when the ball possibly collided with the paddle. Then, by using the paddle's current position and paddle velocity, we are able to find where the paddle previously was, and measure how far the paddle had moved during that time frame. By multiplying the time frame from the ball's collision magnitude to the paddle's change in X position, we are able to find whether the paddle was under the ball at the time of the ball reaching the pixel above the paddle's Y position.

== Testing & Technical Challenges ==

=== VGA Interfacing ===
A DAC resistor ladder was designed and a corresponding custom PCB was acquired to interface the 3.3V digital LCD controller signal with the analog VGA input signal. The VGA cable takes between 0V (completely dark) and 0.7V (maximum brightness) for color pin inputs. The DAC resistor ladder was designed and implemented on a breadboard to verify the design. Validating the design on a breadboard gave us a clear vision during PCB design and planning.

=== VGA Write Timings ===
There were initial tests that were required to be done to see if using a VGA monitor as an output was a viable option for this project. We had to test memory write timings to see if they were fast enough to match the refresh rate of the monitor. Those tests would decide how much of the screen buffer we could write to during the blanking periods during the VGA refresh. This was a good learning experience for testing hardware capabilities and should be done whenever introducing any new aspects to a project.

=== Ball Collision Logic ===
The initial version of the ball collision used pixel values to see if the ball was entering the brick. The problem was that this logic did not account for the case where the ball enters the brick diagonally and the logic for that version of the collision had to be scrapped. The later iteration of the logic was changed to checking whether the ball was potentially crossing into an area that had a brick. This was done by separating the checks into a grid that represented the locations of the blocks. In this case, there was an initial check to see if the ball was crossing a boundary where there might be a new brick. If the ball is crossing the x-boundary and not the y-boundary, then there would be a collision check to see if it collided with a block. If it is crossing the y-boundary and not the x-boundary, once again then there would be a collision check to see if it collided with a block. If it is crossing both the x-boundary and y-boundary, then there would be a check to see which boundary the ball would cross first. This new logic allowed for us to account for the case where the block enters the block diagonally. This taught us that sometimes it is beneficial to scrap a previous iteration and use a new approach to solving a problem because it may lead to better results.

=== Paddle Movement ===
One issue we discovered was the paddle would slow down as it approached the walls. This was due to the difference in "current desired paddle position" and the previous paddle position becoming smaller as the walls were approached, leading to a smaller paddle velocity. To account for this, whenever the accelerometer sent data that mapped to the minimum or maximum paddle index, we would increase the distance of the "current desired paddle position" so the paddle velocity would not slow the paddle down as it approached the walls.

=== Paddle Collision Logic ===
During final integration and testing, one of the biggest issues we encountered was a bug where the ball would not detect a collision with the paddle. The ball would phase right through the paddle instead of bouncing off of it. One of the great ideas that we thought of was to change the color of the ball during each check to see if the checks would go through. This debug check allowed us to verify that section of code that checked for paddle collisions was being skipped even though the ball's next Y position had crossed the paddle's Y position. By having this debugging method, we were able to root cause the issue.

== Conclusion ==
For some of us, this was our first experience in designing and implementing a functional embedded system using FreeRTOS. The challenges we hurdled to accomplish our technical goals helped to shape a journey that was greatly enhanced by the opportunity to meet in person and having a hands-on approach to hardware and component integration. We were able to utilize design principles and implement communication protocols we learned in lecture to pull this project together. Some useful topics include: UART, SPI, I2C, I2S, DMA, FreeRTOS mutexes and binary semaphores, queues, task creation, stack size, task scheduler, interrupt service routines, bootloaders, etc. After extensive hours spent testing and verifying our implementation, we have learned that time allocated for integration and testing will significantly improve chances of a successful project. If we had skipped out on testing and verification, the project could have possibly crashed and burned when it was delivered. We hope this project can be of use to any future engineers following behind us, and we wish you all good times and happy development ahead!

=== Project Video ===
[https://www.youtube.com/watch?v=ipngAGlN86c SJSU CMPE 244 FALL 2021: Embedded Software Project Gameplay]

=== Project Source Code ===
* [https://gitlab.com/fayek.wahhab/sjtwo-c/-/tree/feature/i2s_dma2 Treasure Dive Baseline for Demo Source Code]
* [https://gitlab.com/hobriandh/sjtwo-c/-/tree/final_controller Controller for Demo Source Code]

== References ==
=== Acknowledgement ===
We would like to acknowledge and thank our instructor Preetpal Kang for sharing his knowledge in embedded software development using FreeRTOS.
 
Another shoutout to our art designer Jacob for the unique artwork displayed in our game.

=== Appendix ===
*[https://gitlab.com/sjtwo-c-dev/sjtwo-c/-/blob/master/LPC408X_7X%20User%20Manual.pdf LPC408x/407x Datasheet]
*[https://www.freertos.org/Documentation/RTOS_book.html FreeRTOS Documentation]
*[http://elm-chan.org/fsw/ff/00index_e.html FatFs - Generic FAT Filesystem Module]
*[https://www.digi.com/resources/library/data-sheets/ds_xbeemultipointmodules Xbee Datasheet]
*[http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems Embedded Drivers & Real Time Operating Systems Lessons]

F21: Treasure Dive

2021-12-18T12:48:36Z

Proj user1: /* Objectives & Introduction */

== '''Treasure Dive''' ==
[[File:mermaid.png|thumb|500px|right|Game Screen Capture]]
[[File:Cmpe-244-controller-pic.png|thumb|600px|right|Game Controller]]

== Abstract ==
Treasure Dive is a modern one player game inspired by the popular arcade game Breakout, which was released by Atari in 1976. The ancients rambled about an unconfirmed tale of a shipwreck that saw no survivors. As an adventurous and skilled free diver, you are seeking treasure buried deep in an ocean abyss. You dove down all the way to the seafloor and found the buried treasure but need to make it back to the surface to see another day! The player uses a wireless controller to control an on-screen paddle to break bricks on the top of the screen. When enough bricks are broken, a passage is revealed, allowing you to get closer to the surface.

== Objectives & Introduction ==
The objective of this project is to interface an LPC4088 to a VGA monitor by leveraging the LCD controller and use an accelerometer as the sensor for the player to control the paddle.
 
Two Zigbees RF modules will be integrated for wireless communication between the controller and LPC4088 board via UART protocol.
 
Digital audio output will be enabled by developing an I2S driver and porting an audio tracker library.
 
The General Purpose Direct Memory Access (GPDMA) controller memory-to-memory function will be utilized to decrease memory write times when updating the video buffer.
 
The DMA controller's memory-to-peripheral function will help reduce the frequent I2S interrupts when loading digital audio contents.
 
An SD card peripheral driver to facilitate memory exchange between an SD card will also be implemented.

=== Team Members & Responsibilities ===
[[File:Cmpe-244-brian-pic.png ‎]]
* Brian Ho
** Wiki project schedule planning
** Interfacing the accelerometer
** Digital to Analog Conversion resistor ladder design used to interface digital LCD controller signal with analog VGA port signal
** PCB planning and acquisition
** Procurement of controller hardware
** Wireless communication via Zigbees RF modules and UART protocol
** Wireless controller logic and integration
** Ball movement and wall/brick collision logic
** Hardware/software development and integration
** Final testing and integration
[[File:Cmpe-244-billy-pic.png]]
* [https://www.linkedin.com/in/william-lai-525681139 Billy Lai]
** Wiki project schedule planning
** Increased CPU clock rate from 96 MHz to 120 MHz
** Initial memory write timing and measurements to estimate memory write capacity
** DMA driver memory-to-memory implementation for transferring graphics data to video buffer
** I2S with DMA memory-to-peripheral implementation to reduce memory-write interrupts when I2S Tx FIFO level is low
** Paddle movement and paddle collision logic, testing and integration
** Software development and integration
** Final testing and integration
[[File:Cmpe-244-jasdip-pic.png]]
* Jasdip Sekhon
** Wiki project schedule planning
** PCB design and planning
** File loading method algorithm for storing loaded files in memory
** Audio track and audio effects selection for gameplay audio
** Palette manipulation for gameplay
** Software development and integration
** Final testing and integration
[[File:Cmpe-244-isaac-pic.png ]]
* Isaac Wahhab
** Wiki project schedule planning
** DAC resistor ladder design and integration
** Hardware procurement (LPC4088 board, Zigbees, VGA monitor, etc)
** VGA driving through LCD controller
** Audio tracker library port
** Digital audio output using I2S
** Palette scrolling feature implementation
** Graphics rendering
** Vertical scrolling implementation for the video buffer
** Sprite testing and implementation for the ball
** SD card peripheral driver for reading assets from SD card
** Art assets acquisition
** Hardware/software development and integration
** Gameplay logic design
** Gameplay audio and level design
** Final testing and integration

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
*09/21/2021
|
*09/27/2021
|
* Read previous projects, gather information and discuss among the group members.
* Initial ordering of parts (LPC 4088 board, 64x64 LED screen)
|
* Completed
* Completed
|-
! scope="row"| 2
|
*09/28/2021
|
*10/04/2021
|
* Submit project proposals
|
* Completed
|-
! scope="row"| 3
|
*10/05/2021
|
*10/11/2021
|
* Test if board can drive VGA
* Test varying clock rates
* Hardware cursor on VGA
* Rough display driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 4
|
* 10/12/2021
|
* 10/18/2021
|
* DAC resistor ladder
* RGB channels
* Palette shifting
* Measure memory buffer writing timing
* Wiki schedule planning
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
*10/19/2021
|
*10/25/2021
|
* Calculate upper-bound for VGA refresh timings
* Create GitLab repository for project
* Joystick functionality
* SD card driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
*10/26/2021
|
*11/01/2021
|
* Audio peripheral
* Wireless communication using Zigbee module
* PCB planning
* Accelerometer driver
* Continue SD card driver
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/02/2021

|
* 11/08/2021
|
* Continue audio peripheral
* Continue wireless communication using Zigbee module
* Continue accelerometer driver
* PCB finalizing and ordering
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 11/09/2021
|
* 11/15/2021
|
* Convert PNG data to graphics format
* Game background functionality
* Sprite functionality
* User interface
* Integrate components
* Initial testing
|
* Completed
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 11/16/2021
|
* 11/22/2021
|
* Continue everything from last week
* Game physics (wall collision and AI movement)
* Measurements for wooden case
* Art assets (audio and visual)
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 10
|
*11/23/2021
|
*11/29/2021
|

* Fine-tune user experience
* Make case
|
* Completed
* Completed
|-
! scope="row"| 11
|
* 11/30/2021
|
* 12/06/2021
|
* Continue fine-tune user experience
* Integrate hardware
|
* Completed
* Completed
|-
! scope="row"| 12
|
* 12/07/2021
|
* 12/16/2021
|
* Final testing
* Final demo
|
* Completed
* Completed
|-
|}

== Bill Of Materials ==
{| class="wikitable" style="margin-left: 0px; margin-right: auto;"
|-
! scope="col"| Item#
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| [https://www.embeddedartists.com/products/lpc4088-developers-kit LPC4088-32 Developer's Kit]
| 1
| $300.00
|-
! scope="row"| 2
| Microcontroller Board
| [http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems/page/sj2-board SJ2 Board]
| 1
| $50.00
|-
! scope="row"| 3
| Bluetooth Module
| [https://hub.digi.com/support/products/digi-xbee/digi-xbee-s1 Digi XBee-S1]
| 2
| N/A
|-
! scope="row"| 4
| DAC PCB
| [https://jlcpcb.com JLCPCB (Set of 5)]
| 1
| $15.00
|-
! scope="row"| 4
| Monitor
| Asus TFT Monitor
| 1
| N/A
|-
! scope="row"| 5
| Mini Breadboard
| [https://www.amazon.com/EL-CK-002-Electronic-Breadboard-Capacitor-Potentiometer/dp/B01ERP6WL4/ref=pd_sbs_147_2/132-6936347-2117569?_encoding=UTF8&pd_rd_i=B01ERP6WL4&pd_rd_r=6a783095-ec87-42ae-9f01-a5ebc9a44607&pd_rd_w=V2aU6&pd_rd_wg=11h1i&pf_rd_p=d66372fe-68a6-48a3-90ec-41d7f64212be&pf_rd_r=368WNKXFBBQY2FJQ4119&psc=1&refRID=368WNKXFBBQY2FJQ4119 ELEGOO Mini Breadboard]
| 1
| $13.99
|-
! scope="row"| 6
| Arcade buttons
| [https://www.amazon.com/EG-STARTS-Illuminated-Buttons-Raspberry/dp/B01N11BDX9?pd_rd_w=DiSpt&pf_rd_p=5bc65336-a8b0-4888-9e70-4d4ce2f6bd59&pf_rd_r=JCH2VQYXFPHND4XZ4FAM&pd_rd_r=8d6243dd-e1bd-4dae-9b19-be2e00f3085f&pd_rd_wg=K4lt9&pd_rd_i=B01N11BDX9&psc=1&ref_=pd_bap_d_rp_1_i EG STARTS LED Arcade Buttons]
| 2
| $11.99
|-
! scope="row"| 7
| VGA cable
| N/A
| 1
| N/A
|-
! scope="row"| 7
| SD card
| N/A
| 1
| N/A
|-
|}

== Design & Implementation ==

=== Hardware Design ===
We designed our PCB using Autodesk EAGLE based on the tutorial on PCB design during the class lecture. We ordered our PCB to be manufactured from JLCPCB. The PCB consists of an R-2R resistor ladder DAC for the purpose of converting the digital signal from the microcontroller to analog to display graphics on the VGA monitor. Since the VGA input for each color channel is 0-0.7V, the resistor ladder scaled the LPC4088 outputs into proper DAC values for the VGA to take in. Five bits are passed to each of the red, green, and blue channels of the VGA in addition to other signals, such as Horizontal sync (HSYNC) and Vertical sync (VSYNC).
 
[[File:pcbSchematic.png|500px|thumb|left|PCB Schematic]]
[[File:pcbLayout.png|500px|thumb|right|PCB Layout]]
[[File:pcbPic-min.png|500px|thumb|right|Manufactured PCB]]
[[File:pcbManufactured.png|500px|thumb|left|Soldered PCB]]
 

=== Hardware Interface ===

Two Digi XBee devices were used to communicate between the SJ2-Board controller and the LPC4088 board to control the paddle on the game. The XBee devices were programmed using the XTCU programming application provided by Digi International. The SJ2-Board used one XBee as a transmitted and the LPC4088 used another as a receiver. The two modules communicated using UART frames and a single byte represented any change of movement or button presses sent by the controller.

{| class="wikitable"
|-
! scope="col"| XCTU
! scope="col"| Transmitter
! scope="col"| Receiver
|-
! scope="row"| CH
| C
| C
|-
! scope="row"| ID
| 6969
| 6969
|-
! scope="row"| CE
| Coordinator
| Endpoint
|-
! scope="row"| Baud Rate
| 9600
| 9600
|-
|}
[[File:sj2Board.png|380px|thumb|center|SJ2 Board]]
[[File:CMPE146_F16_MW_XbeeModule.jpg|200px|thumb|center|XBee Device]]
 

Two LED buttons were used to interface with the controller as inputs. They were both set up using GPIO.
[[File:VGA LED Button.jpg|200px|thumb|center|LED Button]]
 
[[File:VGA_Controller_Block_Diagram.PNG|380px|thumb|center|Controller Block Diagram]]
 

==== I2S Peripheral and Stereo Audio Decoder ====

The LPC4088 is equipped with an I2S stereo encoder. This module provides a hardware-enabled solution for sending I2S-encoded audio data packets to stereo decoders, which produce the corresponding audio waveform. For this project, since 16-bit stereo audio playback was chosen, data packets transmitted on this interface were sent in the following format:

[[File:I2S_audio_data_word.png]]

Here, each word of data encodes a single packet of audio in two 16-bit sections: one signal level for the left channel, and another for the right. These 4 bytes are sent at a rate of 44100 Hz, resulting in a total data transmission of 176400 bytes a second. Rather than needing to be software-defined, this transmission is automatically handled by hardware using a 8-word deep FIFO queue that manages streaming out the audio data at a fixed rate, saving a significant amount of CPU cycles.

This encoded audio signal was sent to a dedicated external [https://www.nxp.com/docs/en/data-sheet/UDA1380.pdf stereo decoder], which output to a 3.5 mm audio jack connected to speakers in the VGA monitor.

==== LCD Peripheral ====

The LCD peripheral was configured for 16-bit color palette (5 bits of color data for each color level -- red, green, and blue -- and an unused intensity bit), at a resolution of 800 by 600 pixels, and with an active color palette size of 256 colors. Since the logic for driving an LCD display has the same concepts (horizontal blanking, vertical blanking, etc) and timing information for any given resolution was the same, we were able to directly drive a VGA display with the same signals and timing data (after the digital signal had been converted to analog).

[[File:LCD_%2B_VGA_Video_signal.png]]

The timing register configurations needed to achieve the requested video display was derived from a dedicated website for [http://tinyvga.com/vga-timing/800x600@60Hz VGA signal info].

==== External Memory ====

The game's visual assets and tracker audio file (both after being loaded from an SD card) and the game's video buffer required more memory than what is present natively in the microcontroller. To accommodate this, an external 32MiB memory module was used. This module was interfaced with using the external memory controller peripheral present on the board.

=== Software Design ===
==== Controller ====

There is an internal accelerometer on the SJ2 board that is utilized as the movement controls for the paddle movement. The y-axis value of the magnitude is polled 17 times in 17 milliseconds and is divided by 17 to get an average reading per that time frame. The maximum tilt value that can be read by the controller is set to 700 out of the total 1000 (highest possible value). The current tilt value is divided by 31 to get a total of 32 possible readings to represent 5 bits of magnitude.

The controller sends one byte of data to the console whenever there is any change in magnitude on the accelerometer or button changes. The byte contains all the information needed to control the paddle on the screen. The first two bits on the byte represent 0 if there is no button input and 1 if there is a button input. The third bit represents whether the controller is tilted left or right with 0 being left and 1 being right. The last five bits represent the controller accelerometer's magnitude.

[[File:VGA_UART_Frame.PNG]]

==== Audio ====

Sound design was provided to the project (in the form of music and sound effects) through the use of a [https://https://en.wikipedia.org/wiki/Music_tracker tracker audio library]. Source code was adopted from the [https://github.com/schismtracker/schismtracker Schism Tracker] library and altered with the following modifications:

- All UI logic was stripped from the source code

- All file handling logic was reworked to use the SD card reading functions, as the project does not support POSIX file calls

- The [https://www.libsdl.org/ Simple DirectMedia Layer] hooks were replaced with a FreeRTOS task that periodically called the waveform generation callback function, which was passed to the I2S peripheral hooked up to a stereo decoder.

=== Implementation ===

==== Ball Logic ====

The ball is controlled by the buttons and the controller accelerometer values. When the button is pressed, the ball is initially sent with a random velocity. The velocity is determined by a substate timer and is initially set as one of 6 possible values.

{| class="wikitable"
|-
! scope="col"| Substate Count
! scope="col"| X-Velocity
! scope="col"| Y-Velocity
|-
! scope="row"| 0
| 4
| -2
|-
! scope="row"| 1
| -4
| -2
|-
! scope="row"| 2
| 2
| -4
|-
! scope="row"| 3
| -2
| -4
|-
! scope="row"| 4
| 1
| -5
|-
! scope="row"| 5
| -1
| -5
|-
|}

The ball's velocity changes whenever it detects a collision with either the side of the screen, the bricks on the top of the level, or the paddle at the bottom. If colliding with the paddle, the ball's velocity will take into account the paddle's velocity and add it to its current velocity. This is done by adding the paddle's x-velocity and adjusting the y-velocity by normalizing the velocity vectors to have a magnitude of 5. This means the ball will always be moving with a magnitude of 5 in any direction after it hits the paddle. If the ball collides with anything other than the paddle, then the velocity of the ball is changed with no increase or decrease of magnitude. If the ball collides on the side of a wall or block, the x-velocity will reflect (go from negative to positive or vice-versa) and the y-velocity will remain constant. If the ball collides on the top or bottom of a block, the y-velocity will reflect and the x-velocity will remain constant. If it appears that the ball collides with the side and top or bottom of something, then the ball will check whichever face it will collide with first and then calculate the other face. Lastly, when the ball collides with the bottom of the screen, then the player experiences a "death" and the ball will reset on the paddle. If the ball collides with the top of the screen, the screen will scroll up for the user to play on the next level.

Whenever the ball collides with a brick, the brick will disappear. To accomplish this, there is a 2-D array of unsigned integers that contains the values of the bricks. If the integer is 0, then there is no block. If the integer is any value other than 0, then the block is filled and has collision logic. If the ball approaches a region that might have a block, then there will be a check within that 2-D to see if the block is filled or empty. If the block is filled, there will be the collision logic that changes the vector of the ball and sets the integer value in the 2-D array to 0 to represent that the block has been broken.

==== Paddle Movement ====
The paddle movement is controlled by the accelerometer data being wirelessly transmitted from the SJ-Two board to the LPC4088 board. The accelerometer data will map to 63 possible indexes for the paddle location. Originally, the paddle would jump directly into the corresponding index location depending on the accelerometer input. We later iterated on this process by implementing an algorithm to smoothly transition into these indexes. This was done by calculating a paddle velocity using the "current desired paddle position" and the previous paddle position. We adjusted the parameters of the algorithm iteratively to produce the smooth paddle movement.

==== Paddle Collision Logic ====
The position of the ball is calculated by adding delta_x and delta_y to the ball's current x and y position. For each frame, if the ball's next Y position crosses the paddle's Y position, we would check for a paddle collision. To do this, we used the delta_x and delta_y to get the ball's movement as a slope. We then substituted the slope, the ball's current position, and 1 pixel above the paddle's Y position into a point-slope equation (y2 - y1) = m(x2 - x1) to get the paddle's X position at the time of the ball reaching the pixel above paddle's Y position. After finding the location where the ball is expected to collide with the paddle, we then calculated the ball's collision magnitude and the ball's next location's magnitude. Using a ratio of these two magnitudes, we have a frame time fraction of when the ball possibly collided with the paddle. Then, by using the paddle's current position and paddle velocity, we are able to find where the paddle previously was, and measure how far the paddle had moved during that time frame. By multiplying the time frame from the ball's collision magnitude to the paddle's change in X position, we are able to find whether the paddle was under the ball at the time of the ball reaching the pixel above the paddle's Y position.

== Testing & Technical Challenges ==

=== VGA Interfacing ===
A DAC resistor ladder was designed and a corresponding custom PCB was acquired to interface the 3.3V digital LCD controller signal with the analog VGA input signal. The VGA cable takes between 0V (completely dark) and 0.7V (maximum brightness) for color pin inputs. The DAC resistor ladder was designed and implemented on a breadboard to verify the design. Validating the design on a breadboard gave us a clear vision during PCB design and planning.

=== VGA Write Timings ===
There were initial tests that were required to be done to see if using a VGA monitor as an output was a viable option for this project. We had to test memory write timings to see if they were fast enough to match the refresh rate of the monitor. Those tests would decide how much of the screen buffer we could write to during the blanking periods during the VGA refresh. This was a good learning experience for testing hardware capabilities and should be done whenever introducing any new aspects to a project.

=== Ball Collision Logic ===
The initial version of the ball collision used pixel values to see if the ball was entering the brick. The problem was that this logic did not account for the case where the ball enters the brick diagonally and the logic for that version of the collision had to be scrapped. The later iteration of the logic was changed to checking whether the ball was potentially crossing into an area that had a brick. This was done by separating the checks into a grid that represented the locations of the blocks. In this case, there was an initial check to see if the ball was crossing a boundary where there might be a new brick. If the ball is crossing the x-boundary and not the y-boundary, then there would be a collision check to see if it collided with a block. If it is crossing the y-boundary and not the x-boundary, once again then there would be a collision check to see if it collided with a block. If it is crossing both the x-boundary and y-boundary, then there would be a check to see which boundary the ball would cross first. This new logic allowed for us to account for the case where the block enters the block diagonally. This taught us that sometimes it is beneficial to scrap a previous iteration and use a new approach to solving a problem because it may lead to better results.

=== Paddle Movement ===
One issue we discovered was the paddle would slow down as it approached the walls. This was due to the difference in "current desired paddle position" and the previous paddle position becoming smaller as the walls were approached, leading to a smaller paddle velocity. To account for this, whenever the accelerometer sent data that mapped to the minimum or maximum paddle index, we would increase the distance of the "current desired paddle position" so the paddle velocity would not slow the paddle down as it approached the walls.

=== Paddle Collision Logic ===
During final integration and testing, one of the biggest issues we encountered was a bug where the ball would not detect a collision with the paddle. The ball would phase right through the paddle instead of bouncing off of it. One of the great ideas that we thought of was to change the color of the ball during each check to see if the checks would go through. This debug check allowed us to verify that section of code that checked for paddle collisions was being skipped even though the ball's next Y position had crossed the paddle's Y position. By having this debugging method, we were able to root cause the issue.

== Conclusion ==
For some of us, this was our first experience in designing and implementing a functional embedded system using FreeRTOS. The challenges we hurdled to accomplish our technical goals helped to shape a journey that was greatly enhanced by the opportunity to meet in person and having a hands-on approach to hardware and component integration. We were able to utilize design principles and implement communication protocols we learned in lecture to pull this project together. Some useful topics include: UART, SPI, I2C, I2S, DMA, FreeRTOS mutexes and binary semaphores, queues, task creation, stack size, task scheduler, interrupt service routines, bootloaders, etc. After extensive hours spent testing and verifying our implementation, we have learned that time allocated for integration and testing will significantly improve chances of a successful project. If we had skipped out on testing and verification, the project could have possibly crashed and burned when it was delivered. We hope this project can be of use to any future engineers following behind us, and we wish you all good times and happy development ahead!

=== Project Video ===
[https://www.youtube.com/watch?v=ipngAGlN86c SJSU CMPE 244 FALL 2021: Embedded Software Project Gameplay]

=== Project Source Code ===
* [https://gitlab.com/fayek.wahhab/sjtwo-c/-/tree/feature/i2s_dma2 Treasure Dive Baseline for Demo Source Code]
* [https://gitlab.com/hobriandh/sjtwo-c/-/tree/final_controller Controller for Demo Source Code]

== References ==
=== Acknowledgement ===
We would like to acknowledge and thank our instructor Preetpal Kang for sharing his knowledge in embedded software development using FreeRTOS.
 
Another shoutout to our art designer Jacob for the unique artwork displayed in our game.

=== Appendix ===
*[https://gitlab.com/sjtwo-c-dev/sjtwo-c/-/blob/master/LPC408X_7X%20User%20Manual.pdf LPC408x/407x Datasheet]
*[https://www.freertos.org/Documentation/RTOS_book.html FreeRTOS Documentation]
*[http://elm-chan.org/fsw/ff/00index_e.html FatFs - Generic FAT Filesystem Module]
*[https://www.digi.com/resources/library/data-sheets/ds_xbeemultipointmodules Xbee Datasheet]
*[http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems Embedded Drivers & Real Time Operating Systems Lessons]

F21: Treasure Dive

2021-12-18T12:47:50Z

Proj user1: /* Acknowledgement */

== '''Treasure Dive''' ==
[[File:mermaid.png|thumb|500px|right|Game Screen Capture]]
[[File:Cmpe-244-controller-pic.png|thumb|600px|right|Game Controller]]

== Abstract ==
Treasure Dive is a modern one player game inspired by the popular arcade game Breakout, which was released by Atari in 1976. The ancients rambled about an unconfirmed tale of a shipwreck that saw no survivors. As an adventurous and skilled free diver, you are seeking treasure buried deep in an ocean abyss. You dove down all the way to the seafloor and found the buried treasure but need to make it back to the surface to see another day! The player uses a wireless controller to control an on-screen paddle to break bricks on the top of the screen. When enough bricks are broken, a passage is revealed, allowing you to get closer to the surface.

== Objectives & Introduction ==
The objective of this project is to interface an LPC4088 to a VGA monitor by leveraging the LCD controller and use an accelerometer as the sensor for the player to control the paddle.
Two Zigbees RF modules will be integrated for wireless communication between the controller and LPC4088 board via UART protocol.
Digital audio output will be enabled by developing an I2S driver and porting an audio tracker library.
The General Purpose Direct Memory Access (GPDMA) controller memory-to-memory function will be utilized to decrease memory write times when updating the video buffer.
The DMA controller's memory-to-peripheral function will help reduce the frequent I2S interrupts when loading digital audio contents.
An SD card peripheral driver to facilitate memory exchange between an SD card will also be implemented.

=== Team Members & Responsibilities ===
[[File:Cmpe-244-brian-pic.png ‎]]
* Brian Ho
** Wiki project schedule planning
** Interfacing the accelerometer
** Digital to Analog Conversion resistor ladder design used to interface digital LCD controller signal with analog VGA port signal
** PCB planning and acquisition
** Procurement of controller hardware
** Wireless communication via Zigbees RF modules and UART protocol
** Wireless controller logic and integration
** Ball movement and wall/brick collision logic
** Hardware/software development and integration
** Final testing and integration
[[File:Cmpe-244-billy-pic.png]]
* [https://www.linkedin.com/in/william-lai-525681139 Billy Lai]
** Wiki project schedule planning
** Increased CPU clock rate from 96 MHz to 120 MHz
** Initial memory write timing and measurements to estimate memory write capacity
** DMA driver memory-to-memory implementation for transferring graphics data to video buffer
** I2S with DMA memory-to-peripheral implementation to reduce memory-write interrupts when I2S Tx FIFO level is low
** Paddle movement and paddle collision logic, testing and integration
** Software development and integration
** Final testing and integration
[[File:Cmpe-244-jasdip-pic.png]]
* Jasdip Sekhon
** Wiki project schedule planning
** PCB design and planning
** File loading method algorithm for storing loaded files in memory
** Audio track and audio effects selection for gameplay audio
** Palette manipulation for gameplay
** Software development and integration
** Final testing and integration
[[File:Cmpe-244-isaac-pic.png ]]
* Isaac Wahhab
** Wiki project schedule planning
** DAC resistor ladder design and integration
** Hardware procurement (LPC4088 board, Zigbees, VGA monitor, etc)
** VGA driving through LCD controller
** Audio tracker library port
** Digital audio output using I2S
** Palette scrolling feature implementation
** Graphics rendering
** Vertical scrolling implementation for the video buffer
** Sprite testing and implementation for the ball
** SD card peripheral driver for reading assets from SD card
** Art assets acquisition
** Hardware/software development and integration
** Gameplay logic design
** Gameplay audio and level design
** Final testing and integration

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
*09/21/2021
|
*09/27/2021
|
* Read previous projects, gather information and discuss among the group members.
* Initial ordering of parts (LPC 4088 board, 64x64 LED screen)
|
* Completed
* Completed
|-
! scope="row"| 2
|
*09/28/2021
|
*10/04/2021
|
* Submit project proposals
|
* Completed
|-
! scope="row"| 3
|
*10/05/2021
|
*10/11/2021
|
* Test if board can drive VGA
* Test varying clock rates
* Hardware cursor on VGA
* Rough display driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 4
|
* 10/12/2021
|
* 10/18/2021
|
* DAC resistor ladder
* RGB channels
* Palette shifting
* Measure memory buffer writing timing
* Wiki schedule planning
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
*10/19/2021
|
*10/25/2021
|
* Calculate upper-bound for VGA refresh timings
* Create GitLab repository for project
* Joystick functionality
* SD card driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
*10/26/2021
|
*11/01/2021
|
* Audio peripheral
* Wireless communication using Zigbee module
* PCB planning
* Accelerometer driver
* Continue SD card driver
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/02/2021

|
* 11/08/2021
|
* Continue audio peripheral
* Continue wireless communication using Zigbee module
* Continue accelerometer driver
* PCB finalizing and ordering
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 11/09/2021
|
* 11/15/2021
|
* Convert PNG data to graphics format
* Game background functionality
* Sprite functionality
* User interface
* Integrate components
* Initial testing
|
* Completed
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 11/16/2021
|
* 11/22/2021
|
* Continue everything from last week
* Game physics (wall collision and AI movement)
* Measurements for wooden case
* Art assets (audio and visual)
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 10
|
*11/23/2021
|
*11/29/2021
|

* Fine-tune user experience
* Make case
|
* Completed
* Completed
|-
! scope="row"| 11
|
* 11/30/2021
|
* 12/06/2021
|
* Continue fine-tune user experience
* Integrate hardware
|
* Completed
* Completed
|-
! scope="row"| 12
|
* 12/07/2021
|
* 12/16/2021
|
* Final testing
* Final demo
|
* Completed
* Completed
|-
|}

== Bill Of Materials ==
{| class="wikitable" style="margin-left: 0px; margin-right: auto;"
|-
! scope="col"| Item#
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| [https://www.embeddedartists.com/products/lpc4088-developers-kit LPC4088-32 Developer's Kit]
| 1
| $300.00
|-
! scope="row"| 2
| Microcontroller Board
| [http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems/page/sj2-board SJ2 Board]
| 1
| $50.00
|-
! scope="row"| 3
| Bluetooth Module
| [https://hub.digi.com/support/products/digi-xbee/digi-xbee-s1 Digi XBee-S1]
| 2
| N/A
|-
! scope="row"| 4
| DAC PCB
| [https://jlcpcb.com JLCPCB (Set of 5)]
| 1
| $15.00
|-
! scope="row"| 4
| Monitor
| Asus TFT Monitor
| 1
| N/A
|-
! scope="row"| 5
| Mini Breadboard
| [https://www.amazon.com/EL-CK-002-Electronic-Breadboard-Capacitor-Potentiometer/dp/B01ERP6WL4/ref=pd_sbs_147_2/132-6936347-2117569?_encoding=UTF8&pd_rd_i=B01ERP6WL4&pd_rd_r=6a783095-ec87-42ae-9f01-a5ebc9a44607&pd_rd_w=V2aU6&pd_rd_wg=11h1i&pf_rd_p=d66372fe-68a6-48a3-90ec-41d7f64212be&pf_rd_r=368WNKXFBBQY2FJQ4119&psc=1&refRID=368WNKXFBBQY2FJQ4119 ELEGOO Mini Breadboard]
| 1
| $13.99
|-
! scope="row"| 6
| Arcade buttons
| [https://www.amazon.com/EG-STARTS-Illuminated-Buttons-Raspberry/dp/B01N11BDX9?pd_rd_w=DiSpt&pf_rd_p=5bc65336-a8b0-4888-9e70-4d4ce2f6bd59&pf_rd_r=JCH2VQYXFPHND4XZ4FAM&pd_rd_r=8d6243dd-e1bd-4dae-9b19-be2e00f3085f&pd_rd_wg=K4lt9&pd_rd_i=B01N11BDX9&psc=1&ref_=pd_bap_d_rp_1_i EG STARTS LED Arcade Buttons]
| 2
| $11.99
|-
! scope="row"| 7
| VGA cable
| N/A
| 1
| N/A
|-
! scope="row"| 7
| SD card
| N/A
| 1
| N/A
|-
|}

== Design & Implementation ==

=== Hardware Design ===
We designed our PCB using Autodesk EAGLE based on the tutorial on PCB design during the class lecture. We ordered our PCB to be manufactured from JLCPCB. The PCB consists of an R-2R resistor ladder DAC for the purpose of converting the digital signal from the microcontroller to analog to display graphics on the VGA monitor. Since the VGA input for each color channel is 0-0.7V, the resistor ladder scaled the LPC4088 outputs into proper DAC values for the VGA to take in. Five bits are passed to each of the red, green, and blue channels of the VGA in addition to other signals, such as Horizontal sync (HSYNC) and Vertical sync (VSYNC).
 
[[File:pcbSchematic.png|500px|thumb|left|PCB Schematic]]
[[File:pcbLayout.png|500px|thumb|right|PCB Layout]]
[[File:pcbPic-min.png|500px|thumb|right|Manufactured PCB]]
[[File:pcbManufactured.png|500px|thumb|left|Soldered PCB]]
 

=== Hardware Interface ===

Two Digi XBee devices were used to communicate between the SJ2-Board controller and the LPC4088 board to control the paddle on the game. The XBee devices were programmed using the XTCU programming application provided by Digi International. The SJ2-Board used one XBee as a transmitted and the LPC4088 used another as a receiver. The two modules communicated using UART frames and a single byte represented any change of movement or button presses sent by the controller.

{| class="wikitable"
|-
! scope="col"| XCTU
! scope="col"| Transmitter
! scope="col"| Receiver
|-
! scope="row"| CH
| C
| C
|-
! scope="row"| ID
| 6969
| 6969
|-
! scope="row"| CE
| Coordinator
| Endpoint
|-
! scope="row"| Baud Rate
| 9600
| 9600
|-
|}
[[File:sj2Board.png|380px|thumb|center|SJ2 Board]]
[[File:CMPE146_F16_MW_XbeeModule.jpg|200px|thumb|center|XBee Device]]
 

Two LED buttons were used to interface with the controller as inputs. They were both set up using GPIO.
[[File:VGA LED Button.jpg|200px|thumb|center|LED Button]]
 
[[File:VGA_Controller_Block_Diagram.PNG|380px|thumb|center|Controller Block Diagram]]
 

==== I2S Peripheral and Stereo Audio Decoder ====

The LPC4088 is equipped with an I2S stereo encoder. This module provides a hardware-enabled solution for sending I2S-encoded audio data packets to stereo decoders, which produce the corresponding audio waveform. For this project, since 16-bit stereo audio playback was chosen, data packets transmitted on this interface were sent in the following format:

[[File:I2S_audio_data_word.png]]

Here, each word of data encodes a single packet of audio in two 16-bit sections: one signal level for the left channel, and another for the right. These 4 bytes are sent at a rate of 44100 Hz, resulting in a total data transmission of 176400 bytes a second. Rather than needing to be software-defined, this transmission is automatically handled by hardware using a 8-word deep FIFO queue that manages streaming out the audio data at a fixed rate, saving a significant amount of CPU cycles.

This encoded audio signal was sent to a dedicated external [https://www.nxp.com/docs/en/data-sheet/UDA1380.pdf stereo decoder], which output to a 3.5 mm audio jack connected to speakers in the VGA monitor.

==== LCD Peripheral ====

The LCD peripheral was configured for 16-bit color palette (5 bits of color data for each color level -- red, green, and blue -- and an unused intensity bit), at a resolution of 800 by 600 pixels, and with an active color palette size of 256 colors. Since the logic for driving an LCD display has the same concepts (horizontal blanking, vertical blanking, etc) and timing information for any given resolution was the same, we were able to directly drive a VGA display with the same signals and timing data (after the digital signal had been converted to analog).

[[File:LCD_%2B_VGA_Video_signal.png]]

The timing register configurations needed to achieve the requested video display was derived from a dedicated website for [http://tinyvga.com/vga-timing/800x600@60Hz VGA signal info].

==== External Memory ====

The game's visual assets and tracker audio file (both after being loaded from an SD card) and the game's video buffer required more memory than what is present natively in the microcontroller. To accommodate this, an external 32MiB memory module was used. This module was interfaced with using the external memory controller peripheral present on the board.

=== Software Design ===
==== Controller ====

There is an internal accelerometer on the SJ2 board that is utilized as the movement controls for the paddle movement. The y-axis value of the magnitude is polled 17 times in 17 milliseconds and is divided by 17 to get an average reading per that time frame. The maximum tilt value that can be read by the controller is set to 700 out of the total 1000 (highest possible value). The current tilt value is divided by 31 to get a total of 32 possible readings to represent 5 bits of magnitude.

The controller sends one byte of data to the console whenever there is any change in magnitude on the accelerometer or button changes. The byte contains all the information needed to control the paddle on the screen. The first two bits on the byte represent 0 if there is no button input and 1 if there is a button input. The third bit represents whether the controller is tilted left or right with 0 being left and 1 being right. The last five bits represent the controller accelerometer's magnitude.

[[File:VGA_UART_Frame.PNG]]

==== Audio ====

Sound design was provided to the project (in the form of music and sound effects) through the use of a [https://https://en.wikipedia.org/wiki/Music_tracker tracker audio library]. Source code was adopted from the [https://github.com/schismtracker/schismtracker Schism Tracker] library and altered with the following modifications:

- All UI logic was stripped from the source code

- All file handling logic was reworked to use the SD card reading functions, as the project does not support POSIX file calls

- The [https://www.libsdl.org/ Simple DirectMedia Layer] hooks were replaced with a FreeRTOS task that periodically called the waveform generation callback function, which was passed to the I2S peripheral hooked up to a stereo decoder.

=== Implementation ===

==== Ball Logic ====

The ball is controlled by the buttons and the controller accelerometer values. When the button is pressed, the ball is initially sent with a random velocity. The velocity is determined by a substate timer and is initially set as one of 6 possible values.

{| class="wikitable"
|-
! scope="col"| Substate Count
! scope="col"| X-Velocity
! scope="col"| Y-Velocity
|-
! scope="row"| 0
| 4
| -2
|-
! scope="row"| 1
| -4
| -2
|-
! scope="row"| 2
| 2
| -4
|-
! scope="row"| 3
| -2
| -4
|-
! scope="row"| 4
| 1
| -5
|-
! scope="row"| 5
| -1
| -5
|-
|}

The ball's velocity changes whenever it detects a collision with either the side of the screen, the bricks on the top of the level, or the paddle at the bottom. If colliding with the paddle, the ball's velocity will take into account the paddle's velocity and add it to its current velocity. This is done by adding the paddle's x-velocity and adjusting the y-velocity by normalizing the velocity vectors to have a magnitude of 5. This means the ball will always be moving with a magnitude of 5 in any direction after it hits the paddle. If the ball collides with anything other than the paddle, then the velocity of the ball is changed with no increase or decrease of magnitude. If the ball collides on the side of a wall or block, the x-velocity will reflect (go from negative to positive or vice-versa) and the y-velocity will remain constant. If the ball collides on the top or bottom of a block, the y-velocity will reflect and the x-velocity will remain constant. If it appears that the ball collides with the side and top or bottom of something, then the ball will check whichever face it will collide with first and then calculate the other face. Lastly, when the ball collides with the bottom of the screen, then the player experiences a "death" and the ball will reset on the paddle. If the ball collides with the top of the screen, the screen will scroll up for the user to play on the next level.

Whenever the ball collides with a brick, the brick will disappear. To accomplish this, there is a 2-D array of unsigned integers that contains the values of the bricks. If the integer is 0, then there is no block. If the integer is any value other than 0, then the block is filled and has collision logic. If the ball approaches a region that might have a block, then there will be a check within that 2-D to see if the block is filled or empty. If the block is filled, there will be the collision logic that changes the vector of the ball and sets the integer value in the 2-D array to 0 to represent that the block has been broken.

==== Paddle Movement ====
The paddle movement is controlled by the accelerometer data being wirelessly transmitted from the SJ-Two board to the LPC4088 board. The accelerometer data will map to 63 possible indexes for the paddle location. Originally, the paddle would jump directly into the corresponding index location depending on the accelerometer input. We later iterated on this process by implementing an algorithm to smoothly transition into these indexes. This was done by calculating a paddle velocity using the "current desired paddle position" and the previous paddle position. We adjusted the parameters of the algorithm iteratively to produce the smooth paddle movement.

==== Paddle Collision Logic ====
The position of the ball is calculated by adding delta_x and delta_y to the ball's current x and y position. For each frame, if the ball's next Y position crosses the paddle's Y position, we would check for a paddle collision. To do this, we used the delta_x and delta_y to get the ball's movement as a slope. We then substituted the slope, the ball's current position, and 1 pixel above the paddle's Y position into a point-slope equation (y2 - y1) = m(x2 - x1) to get the paddle's X position at the time of the ball reaching the pixel above paddle's Y position. After finding the location where the ball is expected to collide with the paddle, we then calculated the ball's collision magnitude and the ball's next location's magnitude. Using a ratio of these two magnitudes, we have a frame time fraction of when the ball possibly collided with the paddle. Then, by using the paddle's current position and paddle velocity, we are able to find where the paddle previously was, and measure how far the paddle had moved during that time frame. By multiplying the time frame from the ball's collision magnitude to the paddle's change in X position, we are able to find whether the paddle was under the ball at the time of the ball reaching the pixel above the paddle's Y position.

== Testing & Technical Challenges ==

=== VGA Interfacing ===
A DAC resistor ladder was designed and a corresponding custom PCB was acquired to interface the 3.3V digital LCD controller signal with the analog VGA input signal. The VGA cable takes between 0V (completely dark) and 0.7V (maximum brightness) for color pin inputs. The DAC resistor ladder was designed and implemented on a breadboard to verify the design. Validating the design on a breadboard gave us a clear vision during PCB design and planning.

=== VGA Write Timings ===
There were initial tests that were required to be done to see if using a VGA monitor as an output was a viable option for this project. We had to test memory write timings to see if they were fast enough to match the refresh rate of the monitor. Those tests would decide how much of the screen buffer we could write to during the blanking periods during the VGA refresh. This was a good learning experience for testing hardware capabilities and should be done whenever introducing any new aspects to a project.

=== Ball Collision Logic ===
The initial version of the ball collision used pixel values to see if the ball was entering the brick. The problem was that this logic did not account for the case where the ball enters the brick diagonally and the logic for that version of the collision had to be scrapped. The later iteration of the logic was changed to checking whether the ball was potentially crossing into an area that had a brick. This was done by separating the checks into a grid that represented the locations of the blocks. In this case, there was an initial check to see if the ball was crossing a boundary where there might be a new brick. If the ball is crossing the x-boundary and not the y-boundary, then there would be a collision check to see if it collided with a block. If it is crossing the y-boundary and not the x-boundary, once again then there would be a collision check to see if it collided with a block. If it is crossing both the x-boundary and y-boundary, then there would be a check to see which boundary the ball would cross first. This new logic allowed for us to account for the case where the block enters the block diagonally. This taught us that sometimes it is beneficial to scrap a previous iteration and use a new approach to solving a problem because it may lead to better results.

=== Paddle Movement ===
One issue we discovered was the paddle would slow down as it approached the walls. This was due to the difference in "current desired paddle position" and the previous paddle position becoming smaller as the walls were approached, leading to a smaller paddle velocity. To account for this, whenever the accelerometer sent data that mapped to the minimum or maximum paddle index, we would increase the distance of the "current desired paddle position" so the paddle velocity would not slow the paddle down as it approached the walls.

=== Paddle Collision Logic ===
During final integration and testing, one of the biggest issues we encountered was a bug where the ball would not detect a collision with the paddle. The ball would phase right through the paddle instead of bouncing off of it. One of the great ideas that we thought of was to change the color of the ball during each check to see if the checks would go through. This debug check allowed us to verify that section of code that checked for paddle collisions was being skipped even though the ball's next Y position had crossed the paddle's Y position. By having this debugging method, we were able to root cause the issue.

== Conclusion ==
For some of us, this was our first experience in designing and implementing a functional embedded system using FreeRTOS. The challenges we hurdled to accomplish our technical goals helped to shape a journey that was greatly enhanced by the opportunity to meet in person and having a hands-on approach to hardware and component integration. We were able to utilize design principles and implement communication protocols we learned in lecture to pull this project together. Some useful topics include: UART, SPI, I2C, I2S, DMA, FreeRTOS mutexes and binary semaphores, queues, task creation, stack size, task scheduler, interrupt service routines, bootloaders, etc. After extensive hours spent testing and verifying our implementation, we have learned that time allocated for integration and testing will significantly improve chances of a successful project. If we had skipped out on testing and verification, the project could have possibly crashed and burned when it was delivered. We hope this project can be of use to any future engineers following behind us, and we wish you all good times and happy development ahead!

=== Project Video ===
[https://www.youtube.com/watch?v=ipngAGlN86c SJSU CMPE 244 FALL 2021: Embedded Software Project Gameplay]

=== Project Source Code ===
* [https://gitlab.com/fayek.wahhab/sjtwo-c/-/tree/feature/i2s_dma2 Treasure Dive Baseline for Demo Source Code]
* [https://gitlab.com/hobriandh/sjtwo-c/-/tree/final_controller Controller for Demo Source Code]

== References ==
=== Acknowledgement ===
We would like to acknowledge and thank our instructor Preetpal Kang for sharing his knowledge in embedded software development using FreeRTOS.
 
Another shoutout to our art designer Jacob for the unique artwork displayed in our game.

=== Appendix ===
*[https://gitlab.com/sjtwo-c-dev/sjtwo-c/-/blob/master/LPC408X_7X%20User%20Manual.pdf LPC408x/407x Datasheet]
*[https://www.freertos.org/Documentation/RTOS_book.html FreeRTOS Documentation]
*[http://elm-chan.org/fsw/ff/00index_e.html FatFs - Generic FAT Filesystem Module]
*[https://www.digi.com/resources/library/data-sheets/ds_xbeemultipointmodules Xbee Datasheet]
*[http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems Embedded Drivers & Real Time Operating Systems Lessons]

F21: Treasure Dive

2021-12-18T12:47:30Z

Proj user1: /* Acknowledgement */

== '''Treasure Dive''' ==
[[File:mermaid.png|thumb|500px|right|Game Screen Capture]]
[[File:Cmpe-244-controller-pic.png|thumb|600px|right|Game Controller]]

== Abstract ==
Treasure Dive is a modern one player game inspired by the popular arcade game Breakout, which was released by Atari in 1976. The ancients rambled about an unconfirmed tale of a shipwreck that saw no survivors. As an adventurous and skilled free diver, you are seeking treasure buried deep in an ocean abyss. You dove down all the way to the seafloor and found the buried treasure but need to make it back to the surface to see another day! The player uses a wireless controller to control an on-screen paddle to break bricks on the top of the screen. When enough bricks are broken, a passage is revealed, allowing you to get closer to the surface.

== Objectives & Introduction ==
The objective of this project is to interface an LPC4088 to a VGA monitor by leveraging the LCD controller and use an accelerometer as the sensor for the player to control the paddle.
Two Zigbees RF modules will be integrated for wireless communication between the controller and LPC4088 board via UART protocol.
Digital audio output will be enabled by developing an I2S driver and porting an audio tracker library.
The General Purpose Direct Memory Access (GPDMA) controller memory-to-memory function will be utilized to decrease memory write times when updating the video buffer.
The DMA controller's memory-to-peripheral function will help reduce the frequent I2S interrupts when loading digital audio contents.
An SD card peripheral driver to facilitate memory exchange between an SD card will also be implemented.

=== Team Members & Responsibilities ===
[[File:Cmpe-244-brian-pic.png ‎]]
* Brian Ho
** Wiki project schedule planning
** Interfacing the accelerometer
** Digital to Analog Conversion resistor ladder design used to interface digital LCD controller signal with analog VGA port signal
** PCB planning and acquisition
** Procurement of controller hardware
** Wireless communication via Zigbees RF modules and UART protocol
** Wireless controller logic and integration
** Ball movement and wall/brick collision logic
** Hardware/software development and integration
** Final testing and integration
[[File:Cmpe-244-billy-pic.png]]
* [https://www.linkedin.com/in/william-lai-525681139 Billy Lai]
** Wiki project schedule planning
** Increased CPU clock rate from 96 MHz to 120 MHz
** Initial memory write timing and measurements to estimate memory write capacity
** DMA driver memory-to-memory implementation for transferring graphics data to video buffer
** I2S with DMA memory-to-peripheral implementation to reduce memory-write interrupts when I2S Tx FIFO level is low
** Paddle movement and paddle collision logic, testing and integration
** Software development and integration
** Final testing and integration
[[File:Cmpe-244-jasdip-pic.png]]
* Jasdip Sekhon
** Wiki project schedule planning
** PCB design and planning
** File loading method algorithm for storing loaded files in memory
** Audio track and audio effects selection for gameplay audio
** Palette manipulation for gameplay
** Software development and integration
** Final testing and integration
[[File:Cmpe-244-isaac-pic.png ]]
* Isaac Wahhab
** Wiki project schedule planning
** DAC resistor ladder design and integration
** Hardware procurement (LPC4088 board, Zigbees, VGA monitor, etc)
** VGA driving through LCD controller
** Audio tracker library port
** Digital audio output using I2S
** Palette scrolling feature implementation
** Graphics rendering
** Vertical scrolling implementation for the video buffer
** Sprite testing and implementation for the ball
** SD card peripheral driver for reading assets from SD card
** Art assets acquisition
** Hardware/software development and integration
** Gameplay logic design
** Gameplay audio and level design
** Final testing and integration

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
*09/21/2021
|
*09/27/2021
|
* Read previous projects, gather information and discuss among the group members.
* Initial ordering of parts (LPC 4088 board, 64x64 LED screen)
|
* Completed
* Completed
|-
! scope="row"| 2
|
*09/28/2021
|
*10/04/2021
|
* Submit project proposals
|
* Completed
|-
! scope="row"| 3
|
*10/05/2021
|
*10/11/2021
|
* Test if board can drive VGA
* Test varying clock rates
* Hardware cursor on VGA
* Rough display driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 4
|
* 10/12/2021
|
* 10/18/2021
|
* DAC resistor ladder
* RGB channels
* Palette shifting
* Measure memory buffer writing timing
* Wiki schedule planning
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
*10/19/2021
|
*10/25/2021
|
* Calculate upper-bound for VGA refresh timings
* Create GitLab repository for project
* Joystick functionality
* SD card driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
*10/26/2021
|
*11/01/2021
|
* Audio peripheral
* Wireless communication using Zigbee module
* PCB planning
* Accelerometer driver
* Continue SD card driver
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/02/2021

|
* 11/08/2021
|
* Continue audio peripheral
* Continue wireless communication using Zigbee module
* Continue accelerometer driver
* PCB finalizing and ordering
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 11/09/2021
|
* 11/15/2021
|
* Convert PNG data to graphics format
* Game background functionality
* Sprite functionality
* User interface
* Integrate components
* Initial testing
|
* Completed
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 11/16/2021
|
* 11/22/2021
|
* Continue everything from last week
* Game physics (wall collision and AI movement)
* Measurements for wooden case
* Art assets (audio and visual)
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 10
|
*11/23/2021
|
*11/29/2021
|

* Fine-tune user experience
* Make case
|
* Completed
* Completed
|-
! scope="row"| 11
|
* 11/30/2021
|
* 12/06/2021
|
* Continue fine-tune user experience
* Integrate hardware
|
* Completed
* Completed
|-
! scope="row"| 12
|
* 12/07/2021
|
* 12/16/2021
|
* Final testing
* Final demo
|
* Completed
* Completed
|-
|}

== Bill Of Materials ==
{| class="wikitable" style="margin-left: 0px; margin-right: auto;"
|-
! scope="col"| Item#
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| [https://www.embeddedartists.com/products/lpc4088-developers-kit LPC4088-32 Developer's Kit]
| 1
| $300.00
|-
! scope="row"| 2
| Microcontroller Board
| [http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems/page/sj2-board SJ2 Board]
| 1
| $50.00
|-
! scope="row"| 3
| Bluetooth Module
| [https://hub.digi.com/support/products/digi-xbee/digi-xbee-s1 Digi XBee-S1]
| 2
| N/A
|-
! scope="row"| 4
| DAC PCB
| [https://jlcpcb.com JLCPCB (Set of 5)]
| 1
| $15.00
|-
! scope="row"| 4
| Monitor
| Asus TFT Monitor
| 1
| N/A
|-
! scope="row"| 5
| Mini Breadboard
| [https://www.amazon.com/EL-CK-002-Electronic-Breadboard-Capacitor-Potentiometer/dp/B01ERP6WL4/ref=pd_sbs_147_2/132-6936347-2117569?_encoding=UTF8&pd_rd_i=B01ERP6WL4&pd_rd_r=6a783095-ec87-42ae-9f01-a5ebc9a44607&pd_rd_w=V2aU6&pd_rd_wg=11h1i&pf_rd_p=d66372fe-68a6-48a3-90ec-41d7f64212be&pf_rd_r=368WNKXFBBQY2FJQ4119&psc=1&refRID=368WNKXFBBQY2FJQ4119 ELEGOO Mini Breadboard]
| 1
| $13.99
|-
! scope="row"| 6
| Arcade buttons
| [https://www.amazon.com/EG-STARTS-Illuminated-Buttons-Raspberry/dp/B01N11BDX9?pd_rd_w=DiSpt&pf_rd_p=5bc65336-a8b0-4888-9e70-4d4ce2f6bd59&pf_rd_r=JCH2VQYXFPHND4XZ4FAM&pd_rd_r=8d6243dd-e1bd-4dae-9b19-be2e00f3085f&pd_rd_wg=K4lt9&pd_rd_i=B01N11BDX9&psc=1&ref_=pd_bap_d_rp_1_i EG STARTS LED Arcade Buttons]
| 2
| $11.99
|-
! scope="row"| 7
| VGA cable
| N/A
| 1
| N/A
|-
! scope="row"| 7
| SD card
| N/A
| 1
| N/A
|-
|}

== Design & Implementation ==

=== Hardware Design ===
We designed our PCB using Autodesk EAGLE based on the tutorial on PCB design during the class lecture. We ordered our PCB to be manufactured from JLCPCB. The PCB consists of an R-2R resistor ladder DAC for the purpose of converting the digital signal from the microcontroller to analog to display graphics on the VGA monitor. Since the VGA input for each color channel is 0-0.7V, the resistor ladder scaled the LPC4088 outputs into proper DAC values for the VGA to take in. Five bits are passed to each of the red, green, and blue channels of the VGA in addition to other signals, such as Horizontal sync (HSYNC) and Vertical sync (VSYNC).
 
[[File:pcbSchematic.png|500px|thumb|left|PCB Schematic]]
[[File:pcbLayout.png|500px|thumb|right|PCB Layout]]
[[File:pcbPic-min.png|500px|thumb|right|Manufactured PCB]]
[[File:pcbManufactured.png|500px|thumb|left|Soldered PCB]]
 

=== Hardware Interface ===

Two Digi XBee devices were used to communicate between the SJ2-Board controller and the LPC4088 board to control the paddle on the game. The XBee devices were programmed using the XTCU programming application provided by Digi International. The SJ2-Board used one XBee as a transmitted and the LPC4088 used another as a receiver. The two modules communicated using UART frames and a single byte represented any change of movement or button presses sent by the controller.

{| class="wikitable"
|-
! scope="col"| XCTU
! scope="col"| Transmitter
! scope="col"| Receiver
|-
! scope="row"| CH
| C
| C
|-
! scope="row"| ID
| 6969
| 6969
|-
! scope="row"| CE
| Coordinator
| Endpoint
|-
! scope="row"| Baud Rate
| 9600
| 9600
|-
|}
[[File:sj2Board.png|380px|thumb|center|SJ2 Board]]
[[File:CMPE146_F16_MW_XbeeModule.jpg|200px|thumb|center|XBee Device]]
 

Two LED buttons were used to interface with the controller as inputs. They were both set up using GPIO.
[[File:VGA LED Button.jpg|200px|thumb|center|LED Button]]
 
[[File:VGA_Controller_Block_Diagram.PNG|380px|thumb|center|Controller Block Diagram]]
 

==== I2S Peripheral and Stereo Audio Decoder ====

The LPC4088 is equipped with an I2S stereo encoder. This module provides a hardware-enabled solution for sending I2S-encoded audio data packets to stereo decoders, which produce the corresponding audio waveform. For this project, since 16-bit stereo audio playback was chosen, data packets transmitted on this interface were sent in the following format:

[[File:I2S_audio_data_word.png]]

Here, each word of data encodes a single packet of audio in two 16-bit sections: one signal level for the left channel, and another for the right. These 4 bytes are sent at a rate of 44100 Hz, resulting in a total data transmission of 176400 bytes a second. Rather than needing to be software-defined, this transmission is automatically handled by hardware using a 8-word deep FIFO queue that manages streaming out the audio data at a fixed rate, saving a significant amount of CPU cycles.

This encoded audio signal was sent to a dedicated external [https://www.nxp.com/docs/en/data-sheet/UDA1380.pdf stereo decoder], which output to a 3.5 mm audio jack connected to speakers in the VGA monitor.

==== LCD Peripheral ====

The LCD peripheral was configured for 16-bit color palette (5 bits of color data for each color level -- red, green, and blue -- and an unused intensity bit), at a resolution of 800 by 600 pixels, and with an active color palette size of 256 colors. Since the logic for driving an LCD display has the same concepts (horizontal blanking, vertical blanking, etc) and timing information for any given resolution was the same, we were able to directly drive a VGA display with the same signals and timing data (after the digital signal had been converted to analog).

[[File:LCD_%2B_VGA_Video_signal.png]]

The timing register configurations needed to achieve the requested video display was derived from a dedicated website for [http://tinyvga.com/vga-timing/800x600@60Hz VGA signal info].

==== External Memory ====

The game's visual assets and tracker audio file (both after being loaded from an SD card) and the game's video buffer required more memory than what is present natively in the microcontroller. To accommodate this, an external 32MiB memory module was used. This module was interfaced with using the external memory controller peripheral present on the board.

=== Software Design ===
==== Controller ====

There is an internal accelerometer on the SJ2 board that is utilized as the movement controls for the paddle movement. The y-axis value of the magnitude is polled 17 times in 17 milliseconds and is divided by 17 to get an average reading per that time frame. The maximum tilt value that can be read by the controller is set to 700 out of the total 1000 (highest possible value). The current tilt value is divided by 31 to get a total of 32 possible readings to represent 5 bits of magnitude.

The controller sends one byte of data to the console whenever there is any change in magnitude on the accelerometer or button changes. The byte contains all the information needed to control the paddle on the screen. The first two bits on the byte represent 0 if there is no button input and 1 if there is a button input. The third bit represents whether the controller is tilted left or right with 0 being left and 1 being right. The last five bits represent the controller accelerometer's magnitude.

[[File:VGA_UART_Frame.PNG]]

==== Audio ====

Sound design was provided to the project (in the form of music and sound effects) through the use of a [https://https://en.wikipedia.org/wiki/Music_tracker tracker audio library]. Source code was adopted from the [https://github.com/schismtracker/schismtracker Schism Tracker] library and altered with the following modifications:

- All UI logic was stripped from the source code

- All file handling logic was reworked to use the SD card reading functions, as the project does not support POSIX file calls

- The [https://www.libsdl.org/ Simple DirectMedia Layer] hooks were replaced with a FreeRTOS task that periodically called the waveform generation callback function, which was passed to the I2S peripheral hooked up to a stereo decoder.

=== Implementation ===

==== Ball Logic ====

The ball is controlled by the buttons and the controller accelerometer values. When the button is pressed, the ball is initially sent with a random velocity. The velocity is determined by a substate timer and is initially set as one of 6 possible values.

{| class="wikitable"
|-
! scope="col"| Substate Count
! scope="col"| X-Velocity
! scope="col"| Y-Velocity
|-
! scope="row"| 0
| 4
| -2
|-
! scope="row"| 1
| -4
| -2
|-
! scope="row"| 2
| 2
| -4
|-
! scope="row"| 3
| -2
| -4
|-
! scope="row"| 4
| 1
| -5
|-
! scope="row"| 5
| -1
| -5
|-
|}

The ball's velocity changes whenever it detects a collision with either the side of the screen, the bricks on the top of the level, or the paddle at the bottom. If colliding with the paddle, the ball's velocity will take into account the paddle's velocity and add it to its current velocity. This is done by adding the paddle's x-velocity and adjusting the y-velocity by normalizing the velocity vectors to have a magnitude of 5. This means the ball will always be moving with a magnitude of 5 in any direction after it hits the paddle. If the ball collides with anything other than the paddle, then the velocity of the ball is changed with no increase or decrease of magnitude. If the ball collides on the side of a wall or block, the x-velocity will reflect (go from negative to positive or vice-versa) and the y-velocity will remain constant. If the ball collides on the top or bottom of a block, the y-velocity will reflect and the x-velocity will remain constant. If it appears that the ball collides with the side and top or bottom of something, then the ball will check whichever face it will collide with first and then calculate the other face. Lastly, when the ball collides with the bottom of the screen, then the player experiences a "death" and the ball will reset on the paddle. If the ball collides with the top of the screen, the screen will scroll up for the user to play on the next level.

Whenever the ball collides with a brick, the brick will disappear. To accomplish this, there is a 2-D array of unsigned integers that contains the values of the bricks. If the integer is 0, then there is no block. If the integer is any value other than 0, then the block is filled and has collision logic. If the ball approaches a region that might have a block, then there will be a check within that 2-D to see if the block is filled or empty. If the block is filled, there will be the collision logic that changes the vector of the ball and sets the integer value in the 2-D array to 0 to represent that the block has been broken.

==== Paddle Movement ====
The paddle movement is controlled by the accelerometer data being wirelessly transmitted from the SJ-Two board to the LPC4088 board. The accelerometer data will map to 63 possible indexes for the paddle location. Originally, the paddle would jump directly into the corresponding index location depending on the accelerometer input. We later iterated on this process by implementing an algorithm to smoothly transition into these indexes. This was done by calculating a paddle velocity using the "current desired paddle position" and the previous paddle position. We adjusted the parameters of the algorithm iteratively to produce the smooth paddle movement.

==== Paddle Collision Logic ====
The position of the ball is calculated by adding delta_x and delta_y to the ball's current x and y position. For each frame, if the ball's next Y position crosses the paddle's Y position, we would check for a paddle collision. To do this, we used the delta_x and delta_y to get the ball's movement as a slope. We then substituted the slope, the ball's current position, and 1 pixel above the paddle's Y position into a point-slope equation (y2 - y1) = m(x2 - x1) to get the paddle's X position at the time of the ball reaching the pixel above paddle's Y position. After finding the location where the ball is expected to collide with the paddle, we then calculated the ball's collision magnitude and the ball's next location's magnitude. Using a ratio of these two magnitudes, we have a frame time fraction of when the ball possibly collided with the paddle. Then, by using the paddle's current position and paddle velocity, we are able to find where the paddle previously was, and measure how far the paddle had moved during that time frame. By multiplying the time frame from the ball's collision magnitude to the paddle's change in X position, we are able to find whether the paddle was under the ball at the time of the ball reaching the pixel above the paddle's Y position.

== Testing & Technical Challenges ==

=== VGA Interfacing ===
A DAC resistor ladder was designed and a corresponding custom PCB was acquired to interface the 3.3V digital LCD controller signal with the analog VGA input signal. The VGA cable takes between 0V (completely dark) and 0.7V (maximum brightness) for color pin inputs. The DAC resistor ladder was designed and implemented on a breadboard to verify the design. Validating the design on a breadboard gave us a clear vision during PCB design and planning.

=== VGA Write Timings ===
There were initial tests that were required to be done to see if using a VGA monitor as an output was a viable option for this project. We had to test memory write timings to see if they were fast enough to match the refresh rate of the monitor. Those tests would decide how much of the screen buffer we could write to during the blanking periods during the VGA refresh. This was a good learning experience for testing hardware capabilities and should be done whenever introducing any new aspects to a project.

=== Ball Collision Logic ===
The initial version of the ball collision used pixel values to see if the ball was entering the brick. The problem was that this logic did not account for the case where the ball enters the brick diagonally and the logic for that version of the collision had to be scrapped. The later iteration of the logic was changed to checking whether the ball was potentially crossing into an area that had a brick. This was done by separating the checks into a grid that represented the locations of the blocks. In this case, there was an initial check to see if the ball was crossing a boundary where there might be a new brick. If the ball is crossing the x-boundary and not the y-boundary, then there would be a collision check to see if it collided with a block. If it is crossing the y-boundary and not the x-boundary, once again then there would be a collision check to see if it collided with a block. If it is crossing both the x-boundary and y-boundary, then there would be a check to see which boundary the ball would cross first. This new logic allowed for us to account for the case where the block enters the block diagonally. This taught us that sometimes it is beneficial to scrap a previous iteration and use a new approach to solving a problem because it may lead to better results.

=== Paddle Movement ===
One issue we discovered was the paddle would slow down as it approached the walls. This was due to the difference in "current desired paddle position" and the previous paddle position becoming smaller as the walls were approached, leading to a smaller paddle velocity. To account for this, whenever the accelerometer sent data that mapped to the minimum or maximum paddle index, we would increase the distance of the "current desired paddle position" so the paddle velocity would not slow the paddle down as it approached the walls.

=== Paddle Collision Logic ===
During final integration and testing, one of the biggest issues we encountered was a bug where the ball would not detect a collision with the paddle. The ball would phase right through the paddle instead of bouncing off of it. One of the great ideas that we thought of was to change the color of the ball during each check to see if the checks would go through. This debug check allowed us to verify that section of code that checked for paddle collisions was being skipped even though the ball's next Y position had crossed the paddle's Y position. By having this debugging method, we were able to root cause the issue.

== Conclusion ==
For some of us, this was our first experience in designing and implementing a functional embedded system using FreeRTOS. The challenges we hurdled to accomplish our technical goals helped to shape a journey that was greatly enhanced by the opportunity to meet in person and having a hands-on approach to hardware and component integration. We were able to utilize design principles and implement communication protocols we learned in lecture to pull this project together. Some useful topics include: UART, SPI, I2C, I2S, DMA, FreeRTOS mutexes and binary semaphores, queues, task creation, stack size, task scheduler, interrupt service routines, bootloaders, etc. After extensive hours spent testing and verifying our implementation, we have learned that time allocated for integration and testing will significantly improve chances of a successful project. If we had skipped out on testing and verification, the project could have possibly crashed and burned when it was delivered. We hope this project can be of use to any future engineers following behind us, and we wish you all good times and happy development ahead!

=== Project Video ===
[https://www.youtube.com/watch?v=ipngAGlN86c SJSU CMPE 244 FALL 2021: Embedded Software Project Gameplay]

=== Project Source Code ===
* [https://gitlab.com/fayek.wahhab/sjtwo-c/-/tree/feature/i2s_dma2 Treasure Dive Baseline for Demo Source Code]
* [https://gitlab.com/hobriandh/sjtwo-c/-/tree/final_controller Controller for Demo Source Code]

== References ==
=== Acknowledgement ===
We would like to acknowledge and thank our instructor Preetpal Kang for sharing his knowledge in embedded software development using FreeRTOS.
Another shoutout to our art designer Jacob for the unique artwork displayed in our game.

=== Appendix ===
*[https://gitlab.com/sjtwo-c-dev/sjtwo-c/-/blob/master/LPC408X_7X%20User%20Manual.pdf LPC408x/407x Datasheet]
*[https://www.freertos.org/Documentation/RTOS_book.html FreeRTOS Documentation]
*[http://elm-chan.org/fsw/ff/00index_e.html FatFs - Generic FAT Filesystem Module]
*[https://www.digi.com/resources/library/data-sheets/ds_xbeemultipointmodules Xbee Datasheet]
*[http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems Embedded Drivers & Real Time Operating Systems Lessons]

F21: Treasure Dive

2021-12-18T12:44:33Z

Proj user1: /* Conclusion */

== '''Treasure Dive''' ==
[[File:mermaid.png|thumb|500px|right|Game Screen Capture]]
[[File:Cmpe-244-controller-pic.png|thumb|600px|right|Game Controller]]

== Abstract ==
Treasure Dive is a modern one player game inspired by the popular arcade game Breakout, which was released by Atari in 1976. The ancients rambled about an unconfirmed tale of a shipwreck that saw no survivors. As an adventurous and skilled free diver, you are seeking treasure buried deep in an ocean abyss. You dove down all the way to the seafloor and found the buried treasure but need to make it back to the surface to see another day! The player uses a wireless controller to control an on-screen paddle to break bricks on the top of the screen. When enough bricks are broken, a passage is revealed, allowing you to get closer to the surface.

== Objectives & Introduction ==
The objective of this project is to interface an LPC4088 to a VGA monitor by leveraging the LCD controller and use an accelerometer as the sensor for the player to control the paddle.
Two Zigbees RF modules will be integrated for wireless communication between the controller and LPC4088 board via UART protocol.
Digital audio output will be enabled by developing an I2S driver and porting an audio tracker library.
The General Purpose Direct Memory Access (GPDMA) controller memory-to-memory function will be utilized to decrease memory write times when updating the video buffer.
The DMA controller's memory-to-peripheral function will help reduce the frequent I2S interrupts when loading digital audio contents.
An SD card peripheral driver to facilitate memory exchange between an SD card will also be implemented.

=== Team Members & Responsibilities ===
[[File:Cmpe-244-brian-pic.png ‎]]
* Brian Ho
** Wiki project schedule planning
** Interfacing the accelerometer
** Digital to Analog Conversion resistor ladder design used to interface digital LCD controller signal with analog VGA port signal
** PCB planning and acquisition
** Procurement of controller hardware
** Wireless communication via Zigbees RF modules and UART protocol
** Wireless controller logic and integration
** Ball movement and wall/brick collision logic
** Hardware/software development and integration
** Final testing and integration
[[File:Cmpe-244-billy-pic.png]]
* [https://www.linkedin.com/in/william-lai-525681139 Billy Lai]
** Wiki project schedule planning
** Increased CPU clock rate from 96 MHz to 120 MHz
** Initial memory write timing and measurements to estimate memory write capacity
** DMA driver memory-to-memory implementation for transferring graphics data to video buffer
** I2S with DMA memory-to-peripheral implementation to reduce memory-write interrupts when I2S Tx FIFO level is low
** Paddle movement and paddle collision logic, testing and integration
** Software development and integration
** Final testing and integration
[[File:Cmpe-244-jasdip-pic.png]]
* Jasdip Sekhon
** Wiki project schedule planning
** PCB design and planning
** File loading method algorithm for storing loaded files in memory
** Audio track and audio effects selection for gameplay audio
** Palette manipulation for gameplay
** Software development and integration
** Final testing and integration
[[File:Cmpe-244-isaac-pic.png ]]
* Isaac Wahhab
** Wiki project schedule planning
** DAC resistor ladder design and integration
** Hardware procurement (LPC4088 board, Zigbees, VGA monitor, etc)
** VGA driving through LCD controller
** Audio tracker library port
** Digital audio output using I2S
** Palette scrolling feature implementation
** Graphics rendering
** Vertical scrolling implementation for the video buffer
** Sprite testing and implementation for the ball
** SD card peripheral driver for reading assets from SD card
** Art assets acquisition
** Hardware/software development and integration
** Gameplay logic design
** Gameplay audio and level design
** Final testing and integration

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
*09/21/2021
|
*09/27/2021
|
* Read previous projects, gather information and discuss among the group members.
* Initial ordering of parts (LPC 4088 board, 64x64 LED screen)
|
* Completed
* Completed
|-
! scope="row"| 2
|
*09/28/2021
|
*10/04/2021
|
* Submit project proposals
|
* Completed
|-
! scope="row"| 3
|
*10/05/2021
|
*10/11/2021
|
* Test if board can drive VGA
* Test varying clock rates
* Hardware cursor on VGA
* Rough display driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 4
|
* 10/12/2021
|
* 10/18/2021
|
* DAC resistor ladder
* RGB channels
* Palette shifting
* Measure memory buffer writing timing
* Wiki schedule planning
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
*10/19/2021
|
*10/25/2021
|
* Calculate upper-bound for VGA refresh timings
* Create GitLab repository for project
* Joystick functionality
* SD card driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
*10/26/2021
|
*11/01/2021
|
* Audio peripheral
* Wireless communication using Zigbee module
* PCB planning
* Accelerometer driver
* Continue SD card driver
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/02/2021

|
* 11/08/2021
|
* Continue audio peripheral
* Continue wireless communication using Zigbee module
* Continue accelerometer driver
* PCB finalizing and ordering
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 11/09/2021
|
* 11/15/2021
|
* Convert PNG data to graphics format
* Game background functionality
* Sprite functionality
* User interface
* Integrate components
* Initial testing
|
* Completed
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 11/16/2021
|
* 11/22/2021
|
* Continue everything from last week
* Game physics (wall collision and AI movement)
* Measurements for wooden case
* Art assets (audio and visual)
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 10
|
*11/23/2021
|
*11/29/2021
|

* Fine-tune user experience
* Make case
|
* Completed
* Completed
|-
! scope="row"| 11
|
* 11/30/2021
|
* 12/06/2021
|
* Continue fine-tune user experience
* Integrate hardware
|
* Completed
* Completed
|-
! scope="row"| 12
|
* 12/07/2021
|
* 12/16/2021
|
* Final testing
* Final demo
|
* Completed
* Completed
|-
|}

== Bill Of Materials ==
{| class="wikitable" style="margin-left: 0px; margin-right: auto;"
|-
! scope="col"| Item#
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| [https://www.embeddedartists.com/products/lpc4088-developers-kit LPC4088-32 Developer's Kit]
| 1
| $300.00
|-
! scope="row"| 2
| Microcontroller Board
| [http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems/page/sj2-board SJ2 Board]
| 1
| $50.00
|-
! scope="row"| 3
| Bluetooth Module
| [https://hub.digi.com/support/products/digi-xbee/digi-xbee-s1 Digi XBee-S1]
| 2
| N/A
|-
! scope="row"| 4
| DAC PCB
| [https://jlcpcb.com JLCPCB (Set of 5)]
| 1
| $15.00
|-
! scope="row"| 4
| Monitor
| Asus TFT Monitor
| 1
| N/A
|-
! scope="row"| 5
| Mini Breadboard
| [https://www.amazon.com/EL-CK-002-Electronic-Breadboard-Capacitor-Potentiometer/dp/B01ERP6WL4/ref=pd_sbs_147_2/132-6936347-2117569?_encoding=UTF8&pd_rd_i=B01ERP6WL4&pd_rd_r=6a783095-ec87-42ae-9f01-a5ebc9a44607&pd_rd_w=V2aU6&pd_rd_wg=11h1i&pf_rd_p=d66372fe-68a6-48a3-90ec-41d7f64212be&pf_rd_r=368WNKXFBBQY2FJQ4119&psc=1&refRID=368WNKXFBBQY2FJQ4119 ELEGOO Mini Breadboard]
| 1
| $13.99
|-
! scope="row"| 6
| Arcade buttons
| [https://www.amazon.com/EG-STARTS-Illuminated-Buttons-Raspberry/dp/B01N11BDX9?pd_rd_w=DiSpt&pf_rd_p=5bc65336-a8b0-4888-9e70-4d4ce2f6bd59&pf_rd_r=JCH2VQYXFPHND4XZ4FAM&pd_rd_r=8d6243dd-e1bd-4dae-9b19-be2e00f3085f&pd_rd_wg=K4lt9&pd_rd_i=B01N11BDX9&psc=1&ref_=pd_bap_d_rp_1_i EG STARTS LED Arcade Buttons]
| 2
| $11.99
|-
! scope="row"| 7
| VGA cable
| N/A
| 1
| N/A
|-
! scope="row"| 7
| SD card
| N/A
| 1
| N/A
|-
|}

== Design & Implementation ==

=== Hardware Design ===
We designed our PCB using Autodesk EAGLE based on the tutorial on PCB design during the class lecture. We ordered our PCB to be manufactured from JLCPCB. The PCB consists of an R-2R resistor ladder DAC for the purpose of converting the digital signal from the microcontroller to analog to display graphics on the VGA monitor. Since the VGA input for each color channel is 0-0.7V, the resistor ladder scaled the LPC4088 outputs into proper DAC values for the VGA to take in. Five bits are passed to each of the red, green, and blue channels of the VGA in addition to other signals, such as Horizontal sync (HSYNC) and Vertical sync (VSYNC).
 
[[File:pcbSchematic.png|500px|thumb|left|PCB Schematic]]
[[File:pcbLayout.png|500px|thumb|right|PCB Layout]]
[[File:pcbPic-min.png|500px|thumb|right|Manufactured PCB]]
[[File:pcbManufactured.png|500px|thumb|left|Soldered PCB]]
 

=== Hardware Interface ===

Two Digi XBee devices were used to communicate between the SJ2-Board controller and the LPC4088 board to control the paddle on the game. The XBee devices were programmed using the XTCU programming application provided by Digi International. The SJ2-Board used one XBee as a transmitted and the LPC4088 used another as a receiver. The two modules communicated using UART frames and a single byte represented any change of movement or button presses sent by the controller.

{| class="wikitable"
|-
! scope="col"| XCTU
! scope="col"| Transmitter
! scope="col"| Receiver
|-
! scope="row"| CH
| C
| C
|-
! scope="row"| ID
| 6969
| 6969
|-
! scope="row"| CE
| Coordinator
| Endpoint
|-
! scope="row"| Baud Rate
| 9600
| 9600
|-
|}
[[File:sj2Board.png|380px|thumb|center|SJ2 Board]]
[[File:CMPE146_F16_MW_XbeeModule.jpg|200px|thumb|center|XBee Device]]
 

Two LED buttons were used to interface with the controller as inputs. They were both set up using GPIO.
[[File:VGA LED Button.jpg|200px|thumb|center|LED Button]]
 
[[File:VGA_Controller_Block_Diagram.PNG|380px|thumb|center|Controller Block Diagram]]
 

==== I2S Peripheral and Stereo Audio Decoder ====

The LPC4088 is equipped with an I2S stereo encoder. This module provides a hardware-enabled solution for sending I2S-encoded audio data packets to stereo decoders, which produce the corresponding audio waveform. For this project, since 16-bit stereo audio playback was chosen, data packets transmitted on this interface were sent in the following format:

[[File:I2S_audio_data_word.png]]

Here, each word of data encodes a single packet of audio in two 16-bit sections: one signal level for the left channel, and another for the right. These 4 bytes are sent at a rate of 44100 Hz, resulting in a total data transmission of 176400 bytes a second. Rather than needing to be software-defined, this transmission is automatically handled by hardware using a 8-word deep FIFO queue that manages streaming out the audio data at a fixed rate, saving a significant amount of CPU cycles.

This encoded audio signal was sent to a dedicated external [https://www.nxp.com/docs/en/data-sheet/UDA1380.pdf stereo decoder], which output to a 3.5 mm audio jack connected to speakers in the VGA monitor.

==== LCD Peripheral ====

The LCD peripheral was configured for 16-bit color palette (5 bits of color data for each color level -- red, green, and blue -- and an unused intensity bit), at a resolution of 800 by 600 pixels, and with an active color palette size of 256 colors. Since the logic for driving an LCD display has the same concepts (horizontal blanking, vertical blanking, etc) and timing information for any given resolution was the same, we were able to directly drive a VGA display with the same signals and timing data (after the digital signal had been converted to analog).

[[File:LCD_%2B_VGA_Video_signal.png]]

The timing register configurations needed to achieve the requested video display was derived from a dedicated website for [http://tinyvga.com/vga-timing/800x600@60Hz VGA signal info].

==== External Memory ====

The game's visual assets and tracker audio file (both after being loaded from an SD card) and the game's video buffer required more memory than what is present natively in the microcontroller. To accommodate this, an external 32MiB memory module was used. This module was interfaced with using the external memory controller peripheral present on the board.

=== Software Design ===
==== Controller ====

There is an internal accelerometer on the SJ2 board that is utilized as the movement controls for the paddle movement. The y-axis value of the magnitude is polled 17 times in 17 milliseconds and is divided by 17 to get an average reading per that time frame. The maximum tilt value that can be read by the controller is set to 700 out of the total 1000 (highest possible value). The current tilt value is divided by 31 to get a total of 32 possible readings to represent 5 bits of magnitude.

The controller sends one byte of data to the console whenever there is any change in magnitude on the accelerometer or button changes. The byte contains all the information needed to control the paddle on the screen. The first two bits on the byte represent 0 if there is no button input and 1 if there is a button input. The third bit represents whether the controller is tilted left or right with 0 being left and 1 being right. The last five bits represent the controller accelerometer's magnitude.

[[File:VGA_UART_Frame.PNG]]

==== Audio ====

Sound design was provided to the project (in the form of music and sound effects) through the use of a [https://https://en.wikipedia.org/wiki/Music_tracker tracker audio library]. Source code was adopted from the [https://github.com/schismtracker/schismtracker Schism Tracker] library and altered with the following modifications:

- All UI logic was stripped from the source code

- All file handling logic was reworked to use the SD card reading functions, as the project does not support POSIX file calls

- The [https://www.libsdl.org/ Simple DirectMedia Layer] hooks were replaced with a FreeRTOS task that periodically called the waveform generation callback function, which was passed to the I2S peripheral hooked up to a stereo decoder.

=== Implementation ===

==== Ball Logic ====

The ball is controlled by the buttons and the controller accelerometer values. When the button is pressed, the ball is initially sent with a random velocity. The velocity is determined by a substate timer and is initially set as one of 6 possible values.

{| class="wikitable"
|-
! scope="col"| Substate Count
! scope="col"| X-Velocity
! scope="col"| Y-Velocity
|-
! scope="row"| 0
| 4
| -2
|-
! scope="row"| 1
| -4
| -2
|-
! scope="row"| 2
| 2
| -4
|-
! scope="row"| 3
| -2
| -4
|-
! scope="row"| 4
| 1
| -5
|-
! scope="row"| 5
| -1
| -5
|-
|}

The ball's velocity changes whenever it detects a collision with either the side of the screen, the bricks on the top of the level, or the paddle at the bottom. If colliding with the paddle, the ball's velocity will take into account the paddle's velocity and add it to its current velocity. This is done by adding the paddle's x-velocity and adjusting the y-velocity by normalizing the velocity vectors to have a magnitude of 5. This means the ball will always be moving with a magnitude of 5 in any direction after it hits the paddle. If the ball collides with anything other than the paddle, then the velocity of the ball is changed with no increase or decrease of magnitude. If the ball collides on the side of a wall or block, the x-velocity will reflect (go from negative to positive or vice-versa) and the y-velocity will remain constant. If the ball collides on the top or bottom of a block, the y-velocity will reflect and the x-velocity will remain constant. If it appears that the ball collides with the side and top or bottom of something, then the ball will check whichever face it will collide with first and then calculate the other face. Lastly, when the ball collides with the bottom of the screen, then the player experiences a "death" and the ball will reset on the paddle. If the ball collides with the top of the screen, the screen will scroll up for the user to play on the next level.

Whenever the ball collides with a brick, the brick will disappear. To accomplish this, there is a 2-D array of unsigned integers that contains the values of the bricks. If the integer is 0, then there is no block. If the integer is any value other than 0, then the block is filled and has collision logic. If the ball approaches a region that might have a block, then there will be a check within that 2-D to see if the block is filled or empty. If the block is filled, there will be the collision logic that changes the vector of the ball and sets the integer value in the 2-D array to 0 to represent that the block has been broken.

==== Paddle Movement ====
The paddle movement is controlled by the accelerometer data being wirelessly transmitted from the SJ-Two board to the LPC4088 board. The accelerometer data will map to 63 possible indexes for the paddle location. Originally, the paddle would jump directly into the corresponding index location depending on the accelerometer input. We later iterated on this process by implementing an algorithm to smoothly transition into these indexes. This was done by calculating a paddle velocity using the "current desired paddle position" and the previous paddle position. We adjusted the parameters of the algorithm iteratively to produce the smooth paddle movement.

==== Paddle Collision Logic ====
The position of the ball is calculated by adding delta_x and delta_y to the ball's current x and y position. For each frame, if the ball's next Y position crosses the paddle's Y position, we would check for a paddle collision. To do this, we used the delta_x and delta_y to get the ball's movement as a slope. We then substituted the slope, the ball's current position, and 1 pixel above the paddle's Y position into a point-slope equation (y2 - y1) = m(x2 - x1) to get the paddle's X position at the time of the ball reaching the pixel above paddle's Y position. After finding the location where the ball is expected to collide with the paddle, we then calculated the ball's collision magnitude and the ball's next location's magnitude. Using a ratio of these two magnitudes, we have a frame time fraction of when the ball possibly collided with the paddle. Then, by using the paddle's current position and paddle velocity, we are able to find where the paddle previously was, and measure how far the paddle had moved during that time frame. By multiplying the time frame from the ball's collision magnitude to the paddle's change in X position, we are able to find whether the paddle was under the ball at the time of the ball reaching the pixel above the paddle's Y position.

== Testing & Technical Challenges ==

=== VGA Interfacing ===
A DAC resistor ladder was designed and a corresponding custom PCB was acquired to interface the 3.3V digital LCD controller signal with the analog VGA input signal. The VGA cable takes between 0V (completely dark) and 0.7V (maximum brightness) for color pin inputs. The DAC resistor ladder was designed and implemented on a breadboard to verify the design. Validating the design on a breadboard gave us a clear vision during PCB design and planning.

=== VGA Write Timings ===
There were initial tests that were required to be done to see if using a VGA monitor as an output was a viable option for this project. We had to test memory write timings to see if they were fast enough to match the refresh rate of the monitor. Those tests would decide how much of the screen buffer we could write to during the blanking periods during the VGA refresh. This was a good learning experience for testing hardware capabilities and should be done whenever introducing any new aspects to a project.

=== Ball Collision Logic ===
The initial version of the ball collision used pixel values to see if the ball was entering the brick. The problem was that this logic did not account for the case where the ball enters the brick diagonally and the logic for that version of the collision had to be scrapped. The later iteration of the logic was changed to checking whether the ball was potentially crossing into an area that had a brick. This was done by separating the checks into a grid that represented the locations of the blocks. In this case, there was an initial check to see if the ball was crossing a boundary where there might be a new brick. If the ball is crossing the x-boundary and not the y-boundary, then there would be a collision check to see if it collided with a block. If it is crossing the y-boundary and not the x-boundary, once again then there would be a collision check to see if it collided with a block. If it is crossing both the x-boundary and y-boundary, then there would be a check to see which boundary the ball would cross first. This new logic allowed for us to account for the case where the block enters the block diagonally. This taught us that sometimes it is beneficial to scrap a previous iteration and use a new approach to solving a problem because it may lead to better results.

=== Paddle Movement ===
One issue we discovered was the paddle would slow down as it approached the walls. This was due to the difference in "current desired paddle position" and the previous paddle position becoming smaller as the walls were approached, leading to a smaller paddle velocity. To account for this, whenever the accelerometer sent data that mapped to the minimum or maximum paddle index, we would increase the distance of the "current desired paddle position" so the paddle velocity would not slow the paddle down as it approached the walls.

=== Paddle Collision Logic ===
During final integration and testing, one of the biggest issues we encountered was a bug where the ball would not detect a collision with the paddle. The ball would phase right through the paddle instead of bouncing off of it. One of the great ideas that we thought of was to change the color of the ball during each check to see if the checks would go through. This debug check allowed us to verify that section of code that checked for paddle collisions was being skipped even though the ball's next Y position had crossed the paddle's Y position. By having this debugging method, we were able to root cause the issue.

== Conclusion ==
For some of us, this was our first experience in designing and implementing a functional embedded system using FreeRTOS. The challenges we hurdled to accomplish our technical goals helped to shape a journey that was greatly enhanced by the opportunity to meet in person and having a hands-on approach to hardware and component integration. We were able to utilize design principles and implement communication protocols we learned in lecture to pull this project together. Some useful topics include: UART, SPI, I2C, I2S, DMA, FreeRTOS mutexes and binary semaphores, queues, task creation, stack size, task scheduler, interrupt service routines, bootloaders, etc. After extensive hours spent testing and verifying our implementation, we have learned that time allocated for integration and testing will significantly improve chances of a successful project. If we had skipped out on testing and verification, the project could have possibly crashed and burned when it was delivered. We hope this project can be of use to any future engineers following behind us, and we wish you all good times and happy development ahead!

=== Project Video ===
[https://www.youtube.com/watch?v=ipngAGlN86c SJSU CMPE 244 FALL 2021: Embedded Software Project Gameplay]

=== Project Source Code ===
* [https://gitlab.com/fayek.wahhab/sjtwo-c/-/tree/feature/i2s_dma2 Treasure Dive Baseline for Demo Source Code]
* [https://gitlab.com/hobriandh/sjtwo-c/-/tree/final_controller Controller for Demo Source Code]

== References ==
=== Acknowledgement ===
We would like to acknowledge and thank our instructor Preetpal Kang for sharing his knowledge in embedded software development using FreeRTOS.

=== Appendix ===
*[https://gitlab.com/sjtwo-c-dev/sjtwo-c/-/blob/master/LPC408X_7X%20User%20Manual.pdf LPC408x/407x Datasheet]
*[https://www.freertos.org/Documentation/RTOS_book.html FreeRTOS Documentation]
*[http://elm-chan.org/fsw/ff/00index_e.html FatFs - Generic FAT Filesystem Module]
*[https://www.digi.com/resources/library/data-sheets/ds_xbeemultipointmodules Xbee Datasheet]
*[http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems Embedded Drivers & Real Time Operating Systems Lessons]

F21: Treasure Dive

2021-12-18T12:44:22Z

Proj user1: /* Conclusion */

== '''Treasure Dive''' ==
[[File:mermaid.png|thumb|500px|right|Game Screen Capture]]
[[File:Cmpe-244-controller-pic.png|thumb|600px|right|Game Controller]]

== Abstract ==
Treasure Dive is a modern one player game inspired by the popular arcade game Breakout, which was released by Atari in 1976. The ancients rambled about an unconfirmed tale of a shipwreck that saw no survivors. As an adventurous and skilled free diver, you are seeking treasure buried deep in an ocean abyss. You dove down all the way to the seafloor and found the buried treasure but need to make it back to the surface to see another day! The player uses a wireless controller to control an on-screen paddle to break bricks on the top of the screen. When enough bricks are broken, a passage is revealed, allowing you to get closer to the surface.

== Objectives & Introduction ==
The objective of this project is to interface an LPC4088 to a VGA monitor by leveraging the LCD controller and use an accelerometer as the sensor for the player to control the paddle.
Two Zigbees RF modules will be integrated for wireless communication between the controller and LPC4088 board via UART protocol.
Digital audio output will be enabled by developing an I2S driver and porting an audio tracker library.
The General Purpose Direct Memory Access (GPDMA) controller memory-to-memory function will be utilized to decrease memory write times when updating the video buffer.
The DMA controller's memory-to-peripheral function will help reduce the frequent I2S interrupts when loading digital audio contents.
An SD card peripheral driver to facilitate memory exchange between an SD card will also be implemented.

=== Team Members & Responsibilities ===
[[File:Cmpe-244-brian-pic.png ‎]]
* Brian Ho
** Wiki project schedule planning
** Interfacing the accelerometer
** Digital to Analog Conversion resistor ladder design used to interface digital LCD controller signal with analog VGA port signal
** PCB planning and acquisition
** Procurement of controller hardware
** Wireless communication via Zigbees RF modules and UART protocol
** Wireless controller logic and integration
** Ball movement and wall/brick collision logic
** Hardware/software development and integration
** Final testing and integration
[[File:Cmpe-244-billy-pic.png]]
* [https://www.linkedin.com/in/william-lai-525681139 Billy Lai]
** Wiki project schedule planning
** Increased CPU clock rate from 96 MHz to 120 MHz
** Initial memory write timing and measurements to estimate memory write capacity
** DMA driver memory-to-memory implementation for transferring graphics data to video buffer
** I2S with DMA memory-to-peripheral implementation to reduce memory-write interrupts when I2S Tx FIFO level is low
** Paddle movement and paddle collision logic, testing and integration
** Software development and integration
** Final testing and integration
[[File:Cmpe-244-jasdip-pic.png]]
* Jasdip Sekhon
** Wiki project schedule planning
** PCB design and planning
** File loading method algorithm for storing loaded files in memory
** Audio track and audio effects selection for gameplay audio
** Palette manipulation for gameplay
** Software development and integration
** Final testing and integration
[[File:Cmpe-244-isaac-pic.png ]]
* Isaac Wahhab
** Wiki project schedule planning
** DAC resistor ladder design and integration
** Hardware procurement (LPC4088 board, Zigbees, VGA monitor, etc)
** VGA driving through LCD controller
** Audio tracker library port
** Digital audio output using I2S
** Palette scrolling feature implementation
** Graphics rendering
** Vertical scrolling implementation for the video buffer
** Sprite testing and implementation for the ball
** SD card peripheral driver for reading assets from SD card
** Art assets acquisition
** Hardware/software development and integration
** Gameplay logic design
** Gameplay audio and level design
** Final testing and integration

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
*09/21/2021
|
*09/27/2021
|
* Read previous projects, gather information and discuss among the group members.
* Initial ordering of parts (LPC 4088 board, 64x64 LED screen)
|
* Completed
* Completed
|-
! scope="row"| 2
|
*09/28/2021
|
*10/04/2021
|
* Submit project proposals
|
* Completed
|-
! scope="row"| 3
|
*10/05/2021
|
*10/11/2021
|
* Test if board can drive VGA
* Test varying clock rates
* Hardware cursor on VGA
* Rough display driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 4
|
* 10/12/2021
|
* 10/18/2021
|
* DAC resistor ladder
* RGB channels
* Palette shifting
* Measure memory buffer writing timing
* Wiki schedule planning
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
*10/19/2021
|
*10/25/2021
|
* Calculate upper-bound for VGA refresh timings
* Create GitLab repository for project
* Joystick functionality
* SD card driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
*10/26/2021
|
*11/01/2021
|
* Audio peripheral
* Wireless communication using Zigbee module
* PCB planning
* Accelerometer driver
* Continue SD card driver
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/02/2021

|
* 11/08/2021
|
* Continue audio peripheral
* Continue wireless communication using Zigbee module
* Continue accelerometer driver
* PCB finalizing and ordering
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 11/09/2021
|
* 11/15/2021
|
* Convert PNG data to graphics format
* Game background functionality
* Sprite functionality
* User interface
* Integrate components
* Initial testing
|
* Completed
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 11/16/2021
|
* 11/22/2021
|
* Continue everything from last week
* Game physics (wall collision and AI movement)
* Measurements for wooden case
* Art assets (audio and visual)
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 10
|
*11/23/2021
|
*11/29/2021
|

* Fine-tune user experience
* Make case
|
* Completed
* Completed
|-
! scope="row"| 11
|
* 11/30/2021
|
* 12/06/2021
|
* Continue fine-tune user experience
* Integrate hardware
|
* Completed
* Completed
|-
! scope="row"| 12
|
* 12/07/2021
|
* 12/16/2021
|
* Final testing
* Final demo
|
* Completed
* Completed
|-
|}

== Bill Of Materials ==
{| class="wikitable" style="margin-left: 0px; margin-right: auto;"
|-
! scope="col"| Item#
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| [https://www.embeddedartists.com/products/lpc4088-developers-kit LPC4088-32 Developer's Kit]
| 1
| $300.00
|-
! scope="row"| 2
| Microcontroller Board
| [http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems/page/sj2-board SJ2 Board]
| 1
| $50.00
|-
! scope="row"| 3
| Bluetooth Module
| [https://hub.digi.com/support/products/digi-xbee/digi-xbee-s1 Digi XBee-S1]
| 2
| N/A
|-
! scope="row"| 4
| DAC PCB
| [https://jlcpcb.com JLCPCB (Set of 5)]
| 1
| $15.00
|-
! scope="row"| 4
| Monitor
| Asus TFT Monitor
| 1
| N/A
|-
! scope="row"| 5
| Mini Breadboard
| [https://www.amazon.com/EL-CK-002-Electronic-Breadboard-Capacitor-Potentiometer/dp/B01ERP6WL4/ref=pd_sbs_147_2/132-6936347-2117569?_encoding=UTF8&pd_rd_i=B01ERP6WL4&pd_rd_r=6a783095-ec87-42ae-9f01-a5ebc9a44607&pd_rd_w=V2aU6&pd_rd_wg=11h1i&pf_rd_p=d66372fe-68a6-48a3-90ec-41d7f64212be&pf_rd_r=368WNKXFBBQY2FJQ4119&psc=1&refRID=368WNKXFBBQY2FJQ4119 ELEGOO Mini Breadboard]
| 1
| $13.99
|-
! scope="row"| 6
| Arcade buttons
| [https://www.amazon.com/EG-STARTS-Illuminated-Buttons-Raspberry/dp/B01N11BDX9?pd_rd_w=DiSpt&pf_rd_p=5bc65336-a8b0-4888-9e70-4d4ce2f6bd59&pf_rd_r=JCH2VQYXFPHND4XZ4FAM&pd_rd_r=8d6243dd-e1bd-4dae-9b19-be2e00f3085f&pd_rd_wg=K4lt9&pd_rd_i=B01N11BDX9&psc=1&ref_=pd_bap_d_rp_1_i EG STARTS LED Arcade Buttons]
| 2
| $11.99
|-
! scope="row"| 7
| VGA cable
| N/A
| 1
| N/A
|-
! scope="row"| 7
| SD card
| N/A
| 1
| N/A
|-
|}

== Design & Implementation ==

=== Hardware Design ===
We designed our PCB using Autodesk EAGLE based on the tutorial on PCB design during the class lecture. We ordered our PCB to be manufactured from JLCPCB. The PCB consists of an R-2R resistor ladder DAC for the purpose of converting the digital signal from the microcontroller to analog to display graphics on the VGA monitor. Since the VGA input for each color channel is 0-0.7V, the resistor ladder scaled the LPC4088 outputs into proper DAC values for the VGA to take in. Five bits are passed to each of the red, green, and blue channels of the VGA in addition to other signals, such as Horizontal sync (HSYNC) and Vertical sync (VSYNC).
 
[[File:pcbSchematic.png|500px|thumb|left|PCB Schematic]]
[[File:pcbLayout.png|500px|thumb|right|PCB Layout]]
[[File:pcbPic-min.png|500px|thumb|right|Manufactured PCB]]
[[File:pcbManufactured.png|500px|thumb|left|Soldered PCB]]
 

=== Hardware Interface ===

Two Digi XBee devices were used to communicate between the SJ2-Board controller and the LPC4088 board to control the paddle on the game. The XBee devices were programmed using the XTCU programming application provided by Digi International. The SJ2-Board used one XBee as a transmitted and the LPC4088 used another as a receiver. The two modules communicated using UART frames and a single byte represented any change of movement or button presses sent by the controller.

{| class="wikitable"
|-
! scope="col"| XCTU
! scope="col"| Transmitter
! scope="col"| Receiver
|-
! scope="row"| CH
| C
| C
|-
! scope="row"| ID
| 6969
| 6969
|-
! scope="row"| CE
| Coordinator
| Endpoint
|-
! scope="row"| Baud Rate
| 9600
| 9600
|-
|}
[[File:sj2Board.png|380px|thumb|center|SJ2 Board]]
[[File:CMPE146_F16_MW_XbeeModule.jpg|200px|thumb|center|XBee Device]]
 

Two LED buttons were used to interface with the controller as inputs. They were both set up using GPIO.
[[File:VGA LED Button.jpg|200px|thumb|center|LED Button]]
 
[[File:VGA_Controller_Block_Diagram.PNG|380px|thumb|center|Controller Block Diagram]]
 

==== I2S Peripheral and Stereo Audio Decoder ====

The LPC4088 is equipped with an I2S stereo encoder. This module provides a hardware-enabled solution for sending I2S-encoded audio data packets to stereo decoders, which produce the corresponding audio waveform. For this project, since 16-bit stereo audio playback was chosen, data packets transmitted on this interface were sent in the following format:

[[File:I2S_audio_data_word.png]]

Here, each word of data encodes a single packet of audio in two 16-bit sections: one signal level for the left channel, and another for the right. These 4 bytes are sent at a rate of 44100 Hz, resulting in a total data transmission of 176400 bytes a second. Rather than needing to be software-defined, this transmission is automatically handled by hardware using a 8-word deep FIFO queue that manages streaming out the audio data at a fixed rate, saving a significant amount of CPU cycles.

This encoded audio signal was sent to a dedicated external [https://www.nxp.com/docs/en/data-sheet/UDA1380.pdf stereo decoder], which output to a 3.5 mm audio jack connected to speakers in the VGA monitor.

==== LCD Peripheral ====

The LCD peripheral was configured for 16-bit color palette (5 bits of color data for each color level -- red, green, and blue -- and an unused intensity bit), at a resolution of 800 by 600 pixels, and with an active color palette size of 256 colors. Since the logic for driving an LCD display has the same concepts (horizontal blanking, vertical blanking, etc) and timing information for any given resolution was the same, we were able to directly drive a VGA display with the same signals and timing data (after the digital signal had been converted to analog).

[[File:LCD_%2B_VGA_Video_signal.png]]

The timing register configurations needed to achieve the requested video display was derived from a dedicated website for [http://tinyvga.com/vga-timing/800x600@60Hz VGA signal info].

==== External Memory ====

The game's visual assets and tracker audio file (both after being loaded from an SD card) and the game's video buffer required more memory than what is present natively in the microcontroller. To accommodate this, an external 32MiB memory module was used. This module was interfaced with using the external memory controller peripheral present on the board.

=== Software Design ===
==== Controller ====

There is an internal accelerometer on the SJ2 board that is utilized as the movement controls for the paddle movement. The y-axis value of the magnitude is polled 17 times in 17 milliseconds and is divided by 17 to get an average reading per that time frame. The maximum tilt value that can be read by the controller is set to 700 out of the total 1000 (highest possible value). The current tilt value is divided by 31 to get a total of 32 possible readings to represent 5 bits of magnitude.

The controller sends one byte of data to the console whenever there is any change in magnitude on the accelerometer or button changes. The byte contains all the information needed to control the paddle on the screen. The first two bits on the byte represent 0 if there is no button input and 1 if there is a button input. The third bit represents whether the controller is tilted left or right with 0 being left and 1 being right. The last five bits represent the controller accelerometer's magnitude.

[[File:VGA_UART_Frame.PNG]]

==== Audio ====

Sound design was provided to the project (in the form of music and sound effects) through the use of a [https://https://en.wikipedia.org/wiki/Music_tracker tracker audio library]. Source code was adopted from the [https://github.com/schismtracker/schismtracker Schism Tracker] library and altered with the following modifications:

- All UI logic was stripped from the source code

- All file handling logic was reworked to use the SD card reading functions, as the project does not support POSIX file calls

- The [https://www.libsdl.org/ Simple DirectMedia Layer] hooks were replaced with a FreeRTOS task that periodically called the waveform generation callback function, which was passed to the I2S peripheral hooked up to a stereo decoder.

=== Implementation ===

==== Ball Logic ====

The ball is controlled by the buttons and the controller accelerometer values. When the button is pressed, the ball is initially sent with a random velocity. The velocity is determined by a substate timer and is initially set as one of 6 possible values.

{| class="wikitable"
|-
! scope="col"| Substate Count
! scope="col"| X-Velocity
! scope="col"| Y-Velocity
|-
! scope="row"| 0
| 4
| -2
|-
! scope="row"| 1
| -4
| -2
|-
! scope="row"| 2
| 2
| -4
|-
! scope="row"| 3
| -2
| -4
|-
! scope="row"| 4
| 1
| -5
|-
! scope="row"| 5
| -1
| -5
|-
|}

The ball's velocity changes whenever it detects a collision with either the side of the screen, the bricks on the top of the level, or the paddle at the bottom. If colliding with the paddle, the ball's velocity will take into account the paddle's velocity and add it to its current velocity. This is done by adding the paddle's x-velocity and adjusting the y-velocity by normalizing the velocity vectors to have a magnitude of 5. This means the ball will always be moving with a magnitude of 5 in any direction after it hits the paddle. If the ball collides with anything other than the paddle, then the velocity of the ball is changed with no increase or decrease of magnitude. If the ball collides on the side of a wall or block, the x-velocity will reflect (go from negative to positive or vice-versa) and the y-velocity will remain constant. If the ball collides on the top or bottom of a block, the y-velocity will reflect and the x-velocity will remain constant. If it appears that the ball collides with the side and top or bottom of something, then the ball will check whichever face it will collide with first and then calculate the other face. Lastly, when the ball collides with the bottom of the screen, then the player experiences a "death" and the ball will reset on the paddle. If the ball collides with the top of the screen, the screen will scroll up for the user to play on the next level.

Whenever the ball collides with a brick, the brick will disappear. To accomplish this, there is a 2-D array of unsigned integers that contains the values of the bricks. If the integer is 0, then there is no block. If the integer is any value other than 0, then the block is filled and has collision logic. If the ball approaches a region that might have a block, then there will be a check within that 2-D to see if the block is filled or empty. If the block is filled, there will be the collision logic that changes the vector of the ball and sets the integer value in the 2-D array to 0 to represent that the block has been broken.

==== Paddle Movement ====
The paddle movement is controlled by the accelerometer data being wirelessly transmitted from the SJ-Two board to the LPC4088 board. The accelerometer data will map to 63 possible indexes for the paddle location. Originally, the paddle would jump directly into the corresponding index location depending on the accelerometer input. We later iterated on this process by implementing an algorithm to smoothly transition into these indexes. This was done by calculating a paddle velocity using the "current desired paddle position" and the previous paddle position. We adjusted the parameters of the algorithm iteratively to produce the smooth paddle movement.

==== Paddle Collision Logic ====
The position of the ball is calculated by adding delta_x and delta_y to the ball's current x and y position. For each frame, if the ball's next Y position crosses the paddle's Y position, we would check for a paddle collision. To do this, we used the delta_x and delta_y to get the ball's movement as a slope. We then substituted the slope, the ball's current position, and 1 pixel above the paddle's Y position into a point-slope equation (y2 - y1) = m(x2 - x1) to get the paddle's X position at the time of the ball reaching the pixel above paddle's Y position. After finding the location where the ball is expected to collide with the paddle, we then calculated the ball's collision magnitude and the ball's next location's magnitude. Using a ratio of these two magnitudes, we have a frame time fraction of when the ball possibly collided with the paddle. Then, by using the paddle's current position and paddle velocity, we are able to find where the paddle previously was, and measure how far the paddle had moved during that time frame. By multiplying the time frame from the ball's collision magnitude to the paddle's change in X position, we are able to find whether the paddle was under the ball at the time of the ball reaching the pixel above the paddle's Y position.

== Testing & Technical Challenges ==

=== VGA Interfacing ===
A DAC resistor ladder was designed and a corresponding custom PCB was acquired to interface the 3.3V digital LCD controller signal with the analog VGA input signal. The VGA cable takes between 0V (completely dark) and 0.7V (maximum brightness) for color pin inputs. The DAC resistor ladder was designed and implemented on a breadboard to verify the design. Validating the design on a breadboard gave us a clear vision during PCB design and planning.

=== VGA Write Timings ===
There were initial tests that were required to be done to see if using a VGA monitor as an output was a viable option for this project. We had to test memory write timings to see if they were fast enough to match the refresh rate of the monitor. Those tests would decide how much of the screen buffer we could write to during the blanking periods during the VGA refresh. This was a good learning experience for testing hardware capabilities and should be done whenever introducing any new aspects to a project.

=== Ball Collision Logic ===
The initial version of the ball collision used pixel values to see if the ball was entering the brick. The problem was that this logic did not account for the case where the ball enters the brick diagonally and the logic for that version of the collision had to be scrapped. The later iteration of the logic was changed to checking whether the ball was potentially crossing into an area that had a brick. This was done by separating the checks into a grid that represented the locations of the blocks. In this case, there was an initial check to see if the ball was crossing a boundary where there might be a new brick. If the ball is crossing the x-boundary and not the y-boundary, then there would be a collision check to see if it collided with a block. If it is crossing the y-boundary and not the x-boundary, once again then there would be a collision check to see if it collided with a block. If it is crossing both the x-boundary and y-boundary, then there would be a check to see which boundary the ball would cross first. This new logic allowed for us to account for the case where the block enters the block diagonally. This taught us that sometimes it is beneficial to scrap a previous iteration and use a new approach to solving a problem because it may lead to better results.

=== Paddle Movement ===
One issue we discovered was the paddle would slow down as it approached the walls. This was due to the difference in "current desired paddle position" and the previous paddle position becoming smaller as the walls were approached, leading to a smaller paddle velocity. To account for this, whenever the accelerometer sent data that mapped to the minimum or maximum paddle index, we would increase the distance of the "current desired paddle position" so the paddle velocity would not slow the paddle down as it approached the walls.

=== Paddle Collision Logic ===
During final integration and testing, one of the biggest issues we encountered was a bug where the ball would not detect a collision with the paddle. The ball would phase right through the paddle instead of bouncing off of it. One of the great ideas that we thought of was to change the color of the ball during each check to see if the checks would go through. This debug check allowed us to verify that section of code that checked for paddle collisions was being skipped even though the ball's next Y position had crossed the paddle's Y position. By having this debugging method, we were able to root cause the issue.

== Conclusion ==
For some of us, this was our first experience in designing and implementing a functional embedded system using FreeRTOS. The challenges we hurdled to accomplish our technical goals helped to shape a journey that was greatly enhanced by the opportunity to meet in person and having a hands-on approach to hardware and component integration. We were able to utilize design principles and implement communication protocols we learned in lecture to pull this project together. Some useful topics include: UART, SPI, I2C, I2S, DMA, FreeRTOS mutexes and binary semaphores, queues, task creation, stack size, task scheduler, interrupt service routines, bootloaders, etc. After extensive hours spent testing and verifying our implementation, we have learned that time allocated for integration and testing will significantly improve chances of a successful project. If we had skipped out on testing and verification, the project could have possibly crashed and burned when it was delivered. We hope this project can be of use to any future engineers following behind us, and we wish you all good times and happy development ahead.

=== Project Video ===
[https://www.youtube.com/watch?v=ipngAGlN86c SJSU CMPE 244 FALL 2021: Embedded Software Project Gameplay]

=== Project Source Code ===
* [https://gitlab.com/fayek.wahhab/sjtwo-c/-/tree/feature/i2s_dma2 Treasure Dive Baseline for Demo Source Code]
* [https://gitlab.com/hobriandh/sjtwo-c/-/tree/final_controller Controller for Demo Source Code]

== References ==
=== Acknowledgement ===
We would like to acknowledge and thank our instructor Preetpal Kang for sharing his knowledge in embedded software development using FreeRTOS.

=== Appendix ===
*[https://gitlab.com/sjtwo-c-dev/sjtwo-c/-/blob/master/LPC408X_7X%20User%20Manual.pdf LPC408x/407x Datasheet]
*[https://www.freertos.org/Documentation/RTOS_book.html FreeRTOS Documentation]
*[http://elm-chan.org/fsw/ff/00index_e.html FatFs - Generic FAT Filesystem Module]
*[https://www.digi.com/resources/library/data-sheets/ds_xbeemultipointmodules Xbee Datasheet]
*[http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems Embedded Drivers & Real Time Operating Systems Lessons]

F21: Treasure Dive

2021-12-18T12:19:31Z

Proj user1: /* Team Members & Responsibilities */

== '''Treasure Dive''' ==
[[File:mermaid.png|thumb|500px|right|Game Screen Capture]]
[[File:Cmpe-244-controller-pic.png|thumb|600px|right|Game Controller]]

== Abstract ==
Treasure Dive is a modern one player game inspired by the popular arcade game Breakout, which was released by Atari in 1976. The ancients rambled about an unconfirmed tale of a shipwreck that saw no survivors. As an adventurous and skilled free diver, you are seeking treasure buried deep in an ocean abyss. You dove down all the way to the seafloor and found the buried treasure but need to make it back to the surface to see another day! The player uses a wireless controller to control an on-screen paddle to break bricks on the top of the screen. When enough bricks are broken, a passage is revealed, allowing you to get closer to the surface.

== Objectives & Introduction ==
The objective of this project is to interface an LPC4088 to a VGA monitor by leveraging the LCD controller and use an accelerometer as the sensor for the player to control the paddle.
Two Zigbees RF modules will be integrated for wireless communication between the controller and LPC4088 board via UART protocol.
Digital audio output will be enabled by developing an I2S driver and porting an audio tracker library.
The General Purpose Direct Memory Access (GPDMA) controller memory-to-memory function will be utilized to decrease memory write times when updating the video buffer.
The DMA controller's memory-to-peripheral function will help reduce the frequent I2S interrupts when loading digital audio contents.
An SD card peripheral driver to facilitate memory exchange between an SD card will also be implemented.

=== Team Members & Responsibilities ===
[[File:Cmpe-244-brian-pic.png ‎]]
* Brian Ho
** Wiki project schedule planning
** Interfacing the accelerometer
** Digital to Analog Conversion resistor ladder design used to interface digital LCD controller signal with analog VGA port signal
** PCB planning and acquisition
** Procurement of controller hardware
** Wireless communication via Zigbees RF modules and UART protocol
** Wireless controller logic and integration
** Ball movement and wall/brick collision logic
** Hardware/software development and integration
** Final testing and integration
[[File:Cmpe-244-billy-pic.png]]
* [https://www.linkedin.com/in/william-lai-525681139 Billy Lai]
** Wiki project schedule planning
** Increased CPU clock rate from 96 MHz to 120 MHz
** Initial memory write timing and measurements to estimate memory write capacity
** DMA driver memory-to-memory implementation for transferring graphics data to video buffer
** I2S with DMA memory-to-peripheral implementation to reduce memory-write interrupts when I2S Tx FIFO level is low
** Paddle movement and paddle collision logic, testing and integration
** Software development and integration
** Final testing and integration
[[File:Cmpe-244-jasdip-pic.png]]
* Jasdip Sekhon
** Wiki project schedule planning
** PCB design and planning
** File loading method algorithm for storing loaded files in memory
** Audio track and audio effects selection for gameplay audio
** Palette manipulation for gameplay
** Software development and integration
** Final testing and integration
[[File:Cmpe-244-isaac-pic.png ]]
* Isaac Wahhab
** Wiki project schedule planning
** DAC resistor ladder design and integration
** Hardware procurement (LPC4088 board, Zigbees, VGA monitor, etc)
** VGA driving through LCD controller
** Audio tracker library port
** Digital audio output using I2S
** Palette scrolling feature implementation
** Graphics rendering
** Vertical scrolling implementation for the video buffer
** Sprite testing and implementation for the ball
** SD card peripheral driver for reading assets from SD card
** Art assets acquisition
** Hardware/software development and integration
** Gameplay logic design
** Gameplay audio and level design
** Final testing and integration

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
*09/21/2021
|
*09/27/2021
|
* Read previous projects, gather information and discuss among the group members.
* Initial ordering of parts (LPC 4088 board, 64x64 LED screen)
|
* Completed
* Completed
|-
! scope="row"| 2
|
*09/28/2021
|
*10/04/2021
|
* Submit project proposals
|
* Completed
|-
! scope="row"| 3
|
*10/05/2021
|
*10/11/2021
|
* Test if board can drive VGA
* Test varying clock rates
* Hardware cursor on VGA
* Rough display driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 4
|
* 10/12/2021
|
* 10/18/2021
|
* DAC resistor ladder
* RGB channels
* Palette shifting
* Measure memory buffer writing timing
* Wiki schedule planning
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
*10/19/2021
|
*10/25/2021
|
* Calculate upper-bound for VGA refresh timings
* Create GitLab repository for project
* Joystick functionality
* SD card driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
*10/26/2021
|
*11/01/2021
|
* Audio peripheral
* Wireless communication using Zigbee module
* PCB planning
* Accelerometer driver
* Continue SD card driver
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/02/2021

|
* 11/08/2021
|
* Continue audio peripheral
* Continue wireless communication using Zigbee module
* Continue accelerometer driver
* PCB finalizing and ordering
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 11/09/2021
|
* 11/15/2021
|
* Convert PNG data to graphics format
* Game background functionality
* Sprite functionality
* User interface
* Integrate components
* Initial testing
|
* Completed
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 11/16/2021
|
* 11/22/2021
|
* Continue everything from last week
* Game physics (wall collision and AI movement)
* Measurements for wooden case
* Art assets (audio and visual)
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 10
|
*11/23/2021
|
*11/29/2021
|

* Fine-tune user experience
* Make case
|
* Completed
* Completed
|-
! scope="row"| 11
|
* 11/30/2021
|
* 12/06/2021
|
* Continue fine-tune user experience
* Integrate hardware
|
* Completed
* Completed
|-
! scope="row"| 12
|
* 12/07/2021
|
* 12/16/2021
|
* Final testing
* Final demo
|
* Completed
* Completed
|-
|}

== Bill Of Materials ==
{| class="wikitable" style="margin-left: 0px; margin-right: auto;"
|-
! scope="col"| Item#
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| [https://www.embeddedartists.com/products/lpc4088-developers-kit LPC4088-32 Developer's Kit]
| 1
| $300.00
|-
! scope="row"| 2
| Microcontroller Board
| [http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems/page/sj2-board SJ2 Board]
| 1
| $50.00
|-
! scope="row"| 3
| Bluetooth Module
| [https://hub.digi.com/support/products/digi-xbee/digi-xbee-s1 Digi XBee-S1]
| 2
| N/A
|-
! scope="row"| 4
| DAC PCB
| [https://jlcpcb.com JLCPCB (Set of 5)]
| 1
| $15.00
|-
! scope="row"| 4
| Monitor
| Asus TFT Monitor
| 1
| N/A
|-
! scope="row"| 5
| Mini Breadboard
| [https://www.amazon.com/EL-CK-002-Electronic-Breadboard-Capacitor-Potentiometer/dp/B01ERP6WL4/ref=pd_sbs_147_2/132-6936347-2117569?_encoding=UTF8&pd_rd_i=B01ERP6WL4&pd_rd_r=6a783095-ec87-42ae-9f01-a5ebc9a44607&pd_rd_w=V2aU6&pd_rd_wg=11h1i&pf_rd_p=d66372fe-68a6-48a3-90ec-41d7f64212be&pf_rd_r=368WNKXFBBQY2FJQ4119&psc=1&refRID=368WNKXFBBQY2FJQ4119 ELEGOO Mini Breadboard]
| 1
| $13.99
|-
! scope="row"| 6
| Arcade buttons
| [https://www.amazon.com/EG-STARTS-Illuminated-Buttons-Raspberry/dp/B01N11BDX9?pd_rd_w=DiSpt&pf_rd_p=5bc65336-a8b0-4888-9e70-4d4ce2f6bd59&pf_rd_r=JCH2VQYXFPHND4XZ4FAM&pd_rd_r=8d6243dd-e1bd-4dae-9b19-be2e00f3085f&pd_rd_wg=K4lt9&pd_rd_i=B01N11BDX9&psc=1&ref_=pd_bap_d_rp_1_i EG STARTS LED Arcade Buttons]
| 2
| $11.99
|-
! scope="row"| 7
| VGA cable
| N/A
| 1
| N/A
|-
! scope="row"| 7
| SD card
| N/A
| 1
| N/A
|-
|}

== Design & Implementation ==

=== Hardware Design ===
We designed our PCB using Autodesk EAGLE based on the tutorial on PCB design during the class lecture. We ordered our PCB to be manufactured from JLCPCB. The PCB consists of an R-2R resistor ladder DAC for the purpose of converting the digital signal from the microcontroller to analog to display graphics on the VGA monitor. Since the VGA input for each color channel is 0-0.7V, the resistor ladder scaled the LPC4088 outputs into proper DAC values for the VGA to take in. Five bits are passed to each of the red, green, and blue channels of the VGA in addition to other signals, such as Horizontal sync (HSYNC) and Vertical sync (VSYNC).
 
[[File:pcbSchematic.png|500px|thumb|left|PCB Schematic]]
[[File:pcbLayout.png|500px|thumb|right|PCB Layout]]
[[File:pcbPic-min.png|500px|thumb|right|Manufactured PCB]]
[[File:pcbManufactured.png|500px|thumb|left|Soldered PCB]]
 

=== Hardware Interface ===

Two Digi XBee devices were used to communicate between the SJ2-Board controller and the LPC4088 board to control the paddle on the game. The XBee devices were programmed using the XTCU programming application provided by Digi International. The SJ2-Board used one XBee as a transmitted and the LPC4088 used another as a receiver. The two modules communicated using UART frames and a single byte represented any change of movement or button presses sent by the controller.

{| class="wikitable"
|-
! scope="col"| XCTU
! scope="col"| Transmitter
! scope="col"| Receiver
|-
! scope="row"| CH
| C
| C
|-
! scope="row"| ID
| 6969
| 6969
|-
! scope="row"| CE
| Coordinator
| Endpoint
|-
! scope="row"| Baud Rate
| 9600
| 9600
|-
|}
[[File:sj2Board.png|380px|thumb|center|SJ2 Board]]
[[File:CMPE146_F16_MW_XbeeModule.jpg|200px|thumb|center|XBee Device]]
 

Two LED buttons were used to interface with the controller as inputs. They were both set up using GPIO.
[[File:VGA LED Button.jpg|200px|thumb|center|LED Button]]
 
[[File:VGA_Controller_Block_Diagram.PNG|380px|thumb|center|Controller Block Diagram]]
 

==== I2S Peripheral and Stereo Audio Decoder ====

The LPC4088 is equipped with an I2S stereo encoder. This module provides a hardware-enabled solution for sending I2S-encoded audio data packets to stereo decoders, which produce the corresponding audio waveform. For this project, since 16-bit stereo audio playback was chosen, data packets transmitted on this interface were sent in the following format:

[[File:I2S_audio_data_word.png]]

Here, each word of data encodes a single packet of audio in two 16-bit sections: one signal level for the left channel, and another for the right. These 4 bytes are sent at a rate of 44100 Hz, resulting in a total data transmission of 176400 bytes a second. Rather than needing to be software-defined, this transmission is automatically handled by hardware using a 8-word deep FIFO queue that manages streaming out the audio data at a fixed rate, saving a significant amount of CPU cycles.

This encoded audio signal was sent to a dedicated external [https://www.nxp.com/docs/en/data-sheet/UDA1380.pdf stereo decoder], which output to a 3.5 mm audio jack connected to speakers in the VGA monitor.

==== LCD Peripheral ====

The LCD peripheral was configured for 16-bit color palette (5 bits of color data for each color level -- red, green, and blue -- and an unused intensity bit), at a resolution of 800 by 600 pixels, and with an active color palette size of 256 colors. Since the logic for driving an LCD display has the same concepts (horizontal blanking, vertical blanking, etc) and timing information for any given resolution was the same, we were able to directly drive a VGA display with the same signals and timing data (after the digital signal had been converted to analog).

[[File:LCD_%2B_VGA_Video_signal.png]]

The timing register configurations needed to achieve the requested video display was derived from a dedicated website for [http://tinyvga.com/vga-timing/800x600@60Hz VGA signal info].

==== External Memory ====

The game's visual assets and tracker audio file (both after being loaded from an SD card) and the game's video buffer required more memory than what is present natively in the microcontroller. To accommodate this, an external 32MiB memory module was used. This module was interfaced with using the external memory controller peripheral present on the board.

=== Software Design ===
==== Controller ====

There is an internal accelerometer on the SJ2 board that is utilized as the movement controls for the paddle movement. The y-axis value of the magnitude is polled 17 times in 17 milliseconds and is divided by 17 to get an average reading per that time frame. The maximum tilt value that can be read by the controller is set to 700 out of the total 1000 (highest possible value). The current tilt value is divided by 31 to get a total of 32 possible readings to represent 5 bits of magnitude.

The controller sends one byte of data to the console whenever there is any change in magnitude on the accelerometer or button changes. The byte contains all the information needed to control the paddle on the screen. The first two bits on the byte represent 0 if there is no button input and 1 if there is a button input. The third bit represents whether the controller is tilted left or right with 0 being left and 1 being right. The last five bits represent the controller accelerometer's magnitude.

[[File:VGA_UART_Frame.PNG]]

==== Audio ====

Sound design was provided to the project (in the form of music and sound effects) through the use of a [https://https://en.wikipedia.org/wiki/Music_tracker tracker audio library]. Source code was adopted from the [https://github.com/schismtracker/schismtracker Schism Tracker] library and altered with the following modifications:

- All UI logic was stripped from the source code

- All file handling logic was reworked to use the SD card reading functions, as the project does not support POSIX file calls

- The [https://www.libsdl.org/ Simple DirectMedia Layer] hooks were replaced with a FreeRTOS task that periodically called the waveform generation callback function, which was passed to the I2S peripheral hooked up to a stereo decoder.

=== Implementation ===

==== Ball Logic ====

The ball is controlled by the buttons and the controller accelerometer values. When the button is pressed, the ball is initially sent with a random velocity. The velocity is determined by a substate timer and is initially set as one of 6 possible values.

{| class="wikitable"
|-
! scope="col"| Substate Count
! scope="col"| X-Velocity
! scope="col"| Y-Velocity
|-
! scope="row"| 0
| 4
| -2
|-
! scope="row"| 1
| -4
| -2
|-
! scope="row"| 2
| 2
| -4
|-
! scope="row"| 3
| -2
| -4
|-
! scope="row"| 4
| 1
| -5
|-
! scope="row"| 5
| -1
| -5
|-
|}

The ball's velocity changes whenever it detects a collision with either the side of the screen, the bricks on the top of the level, or the paddle at the bottom. If colliding with the paddle, the ball's velocity will take into account the paddle's velocity and add it to its current velocity. This is done by adding the paddle's x-velocity and adjusting the y-velocity by normalizing the velocity vectors to have a magnitude of 5. This means the ball will always be moving with a magnitude of 5 in any direction after it hits the paddle. If the ball collides with anything other than the paddle, then the velocity of the ball is changed with no increase or decrease of magnitude. If the ball collides on the side of a wall or block, the x-velocity will reflect (go from negative to positive or vice-versa) and the y-velocity will remain constant. If the ball collides on the top or bottom of a block, the y-velocity will reflect and the x-velocity will remain constant. If it appears that the ball collides with the side and top or bottom of something, then the ball will check whichever face it will collide with first and then calculate the other face. Lastly, when the ball collides with the bottom of the screen, then the player experiences a "death" and the ball will reset on the paddle. If the ball collides with the top of the screen, the screen will scroll up for the user to play on the next level.

Whenever the ball collides with a brick, the brick will disappear. To accomplish this, there is a 2-D array of unsigned integers that contains the values of the bricks. If the integer is 0, then there is no block. If the integer is any value other than 0, then the block is filled and has collision logic. If the ball approaches a region that might have a block, then there will be a check within that 2-D to see if the block is filled or empty. If the block is filled, there will be the collision logic that changes the vector of the ball and sets the integer value in the 2-D array to 0 to represent that the block has been broken.

==== Paddle Movement ====
The paddle movement is controlled by the accelerometer data being wirelessly transmitted from the SJ-Two board to the LPC4088 board. The accelerometer data will map to 63 possible indexes for the paddle location. Originally, the paddle would jump directly into the corresponding index location depending on the accelerometer input. We later iterated on this process by implementing an algorithm to smoothly transition into these indexes. This was done by calculating a paddle velocity using the "current desired paddle position" and the previous paddle position. We adjusted the parameters of the algorithm iteratively to produce the smooth paddle movement.

==== Paddle Collision Logic ====
The position of the ball is calculated by adding delta_x and delta_y to the ball's current x and y position. For each frame, if the ball's next Y position crosses the paddle's Y position, we would check for a paddle collision. To do this, we used the delta_x and delta_y to get the ball's movement as a slope. We then substituted the slope, the ball's current position, and 1 pixel above the paddle's Y position into a point-slope equation (y2 - y1) = m(x2 - x1) to get the paddle's X position at the time of the ball reaching the pixel above paddle's Y position. After finding the location where the ball is expected to collide with the paddle, we then calculated the ball's collision magnitude and the ball's next location's magnitude. Using a ratio of these two magnitudes, we have a frame time fraction of when the ball possibly collided with the paddle. Then, by using the paddle's current position and paddle velocity, we are able to find where the paddle previously was, and measure how far the paddle had moved during that time frame. By multiplying the time frame from the ball's collision magnitude to the paddle's change in X position, we are able to find whether the paddle was under the ball at the time of the ball reaching the pixel above the paddle's Y position.

== Testing & Technical Challenges ==

=== VGA Interfacing ===
A DAC resistor ladder was designed and a corresponding custom PCB was acquired to interface the 3.3V digital LCD controller signal with the analog VGA input signal. The VGA cable takes between 0V (completely dark) and 0.7V (maximum brightness) for color pin inputs. The DAC resistor ladder was designed and implemented on a breadboard to verify the design. Validating the design on a breadboard gave us a clear vision during PCB design and planning.

=== VGA Write Timings ===
There were initial tests that were required to be done to see if using a VGA monitor as an output was a viable option for this project. We had to test memory write timings to see if they were fast enough to match the refresh rate of the monitor. Those tests would decide how much of the screen buffer we could write to during the blanking periods during the VGA refresh. This was a good learning experience for testing hardware capabilities and should be done whenever introducing any new aspects to a project.

=== Ball Collision Logic ===
The initial version of the ball collision used pixel values to see if the ball was entering the brick. The problem was that this logic did not account for the case where the ball enters the brick diagonally and the logic for that version of the collision had to be scrapped. The later iteration of the logic was changed to checking whether the ball was potentially crossing into an area that had a brick. This was done by separating the checks into a grid that represented the locations of the blocks. In this case, there was an initial check to see if the ball was crossing a boundary where there might be a new brick. If the ball is crossing the x-boundary and not the y-boundary, then there would be a collision check to see if it collided with a block. If it is crossing the y-boundary and not the x-boundary, once again then there would be a collision check to see if it collided with a block. If it is crossing both the x-boundary and y-boundary, then there would be a check to see which boundary the ball would cross first. This new logic allowed for us to account for the case where the block enters the block diagonally. This taught us that sometimes it is beneficial to scrap a previous iteration and use a new approach to solving a problem because it may lead to better results.

=== Paddle Movement ===
One issue we discovered was the paddle would slow down as it approached the walls. This was due to the difference in "current desired paddle position" and the previous paddle position becoming smaller as the walls were approached, leading to a smaller paddle velocity. To account for this, whenever the accelerometer sent data that mapped to the minimum or maximum paddle index, we would increase the distance of the "current desired paddle position" so the paddle velocity would not slow the paddle down as it approached the walls.

=== Paddle Collision Logic ===
During final integration and testing, one of the biggest issues we encountered was a bug where the ball would not detect a collision with the paddle. The ball would phase right through the paddle instead of bouncing off of it. One of the great ideas that we thought of was to change the color of the ball during each check to see if the checks would go through. This debug check allowed us to verify that section of code that checked for paddle collisions was being skipped even though the ball's next Y position had crossed the paddle's Y position. By having this debugging method, we were able to root cause the issue.

== Conclusion ==
Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

=== Project Video ===
[https://www.youtube.com/watch?v=ipngAGlN86c SJSU CMPE 244 FALL 2021: Embedded Software Project Gameplay]

=== Project Source Code ===
* [https://gitlab.com/fayek.wahhab/sjtwo-c/-/tree/feature/i2s_dma2 Treasure Dive Baseline for Demo Source Code]
* [https://gitlab.com/hobriandh/sjtwo-c/-/tree/final_controller Controller for Demo Source Code]

== References ==
=== Acknowledgement ===
We would like to acknowledge and thank our instructor Preetpal Kang for sharing his knowledge in embedded software development using FreeRTOS.

=== Appendix ===
*[https://gitlab.com/sjtwo-c-dev/sjtwo-c/-/blob/master/LPC408X_7X%20User%20Manual.pdf LPC408x/407x Datasheet]
*[https://www.freertos.org/Documentation/RTOS_book.html FreeRTOS Documentation]
*[http://elm-chan.org/fsw/ff/00index_e.html FatFs - Generic FAT Filesystem Module]
*[https://www.digi.com/resources/library/data-sheets/ds_xbeemultipointmodules Xbee Datasheet]
*[http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems Embedded Drivers & Real Time Operating Systems Lessons]

File:Cmpe-244-billy-pic.png

2021-12-18T12:17:19Z

Proj user1: Proj user1 uploaded a new version of File:Cmpe-244-billy-pic.png

File:Cmpe-244-billy-pic.png

2021-12-18T12:16:17Z

Proj user1: Proj user1 uploaded a new version of File:Cmpe-244-billy-pic.png

F21: Treasure Dive

2021-12-18T12:15:26Z

Proj user1: /* Team Members & Responsibilities */

== '''Treasure Dive''' ==
[[File:mermaid.png|thumb|500px|right|Game Screen Capture]]
[[File:Cmpe-244-controller-pic.png|thumb|600px|right|Game Controller]]

== Abstract ==
Treasure Dive is a modern one player game inspired by the popular arcade game Breakout, which was released by Atari in 1976. The ancients rambled about an unconfirmed tale of a shipwreck that saw no survivors. As an adventurous and skilled free diver, you are seeking treasure buried deep in an ocean abyss. You dove down all the way to the seafloor and found the buried treasure but need to make it back to the surface to see another day! The player uses a wireless controller to control an on-screen paddle to break bricks on the top of the screen. When enough bricks are broken, a passage is revealed, allowing you to get closer to the surface.

== Objectives & Introduction ==
The objective of this project is to interface an LPC4088 to a VGA monitor by leveraging the LCD controller and use an accelerometer as the sensor for the player to control the paddle.
Two Zigbees RF modules will be integrated for wireless communication between the controller and LPC4088 board via UART protocol.
Digital audio output will be enabled by developing an I2S driver and porting an audio tracker library.
The General Purpose Direct Memory Access (GPDMA) controller memory-to-memory function will be utilized to decrease memory write times when updating the video buffer.
The DMA controller's memory-to-peripheral function will help reduce the frequent I2S interrupts when loading digital audio contents.
An SD card peripheral driver to facilitate memory exchange between an SD card will also be implemented.

=== Team Members & Responsibilities ===
* [[File:Cmpe-244-brian-pic.png ‎]]
* Brian Ho
** Wiki project schedule planning
** Interfacing the accelerometer
** Digital to Analog Conversion resistor ladder design used to interface digital LCD controller signal with analog VGA port signal
** PCB planning and acquisition
** Procurement of controller hardware
** Wireless communication via Zigbees RF modules and UART protocol
** Wireless controller logic and integration
** Ball movement and wall/brick collision logic
** Hardware/software development and integration
** Final testing and integration
* [[File:Cmpe-244-billy-pic.png]]
* [https://www.linkedin.com/in/william-lai-525681139 Billy Lai]
** Wiki project schedule planning
** Increased CPU clock rate from 96 MHz to 120 MHz
** Initial memory write timing and measurements to estimate memory write capacity
** DMA driver memory-to-memory implementation for transferring graphics data to video buffer
** I2S with DMA memory-to-peripheral implementation to reduce memory-write interrupts when I2S Tx FIFO level is low
** Paddle movement and paddle collision logic, testing and integration
** Software development and integration
** Final testing and integration
* [[File:Cmpe-244-jasdip-pic.png]]
* Jasdip Sekhon
** Wiki project schedule planning
** PCB design and planning
** File loading method algorithm for storing loaded files in memory
** Audio track and audio effects selection for gameplay audio
** Palette manipulation for gameplay
** Software development and integration
** Final testing and integration
* [[File:Cmpe-244-isaac-pic.png ]]
* Isaac Wahhab
** Wiki project schedule planning
** DAC resistor ladder design and integration
** Hardware procurement (LPC4088 board, Zigbees, VGA monitor, etc)
** VGA driving through LCD controller
** Audio tracker library port
** Digital audio output using I2S
** Palette scrolling feature implementation
** Graphics rendering
** Vertical scrolling implementation for the video buffer
** Sprite testing and implementation for the ball
** SD card peripheral driver for reading assets from SD card
** Art assets acquisition
** Hardware/software development and integration
** Gameplay logic design
** Gameplay audio and level design
** Final testing and integration

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
*09/21/2021
|
*09/27/2021
|
* Read previous projects, gather information and discuss among the group members.
* Initial ordering of parts (LPC 4088 board, 64x64 LED screen)
|
* Completed
* Completed
|-
! scope="row"| 2
|
*09/28/2021
|
*10/04/2021
|
* Submit project proposals
|
* Completed
|-
! scope="row"| 3
|
*10/05/2021
|
*10/11/2021
|
* Test if board can drive VGA
* Test varying clock rates
* Hardware cursor on VGA
* Rough display driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 4
|
* 10/12/2021
|
* 10/18/2021
|
* DAC resistor ladder
* RGB channels
* Palette shifting
* Measure memory buffer writing timing
* Wiki schedule planning
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
*10/19/2021
|
*10/25/2021
|
* Calculate upper-bound for VGA refresh timings
* Create GitLab repository for project
* Joystick functionality
* SD card driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
*10/26/2021
|
*11/01/2021
|
* Audio peripheral
* Wireless communication using Zigbee module
* PCB planning
* Accelerometer driver
* Continue SD card driver
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/02/2021

|
* 11/08/2021
|
* Continue audio peripheral
* Continue wireless communication using Zigbee module
* Continue accelerometer driver
* PCB finalizing and ordering
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 11/09/2021
|
* 11/15/2021
|
* Convert PNG data to graphics format
* Game background functionality
* Sprite functionality
* User interface
* Integrate components
* Initial testing
|
* Completed
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 11/16/2021
|
* 11/22/2021
|
* Continue everything from last week
* Game physics (wall collision and AI movement)
* Measurements for wooden case
* Art assets (audio and visual)
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 10
|
*11/23/2021
|
*11/29/2021
|

* Fine-tune user experience
* Make case
|
* Completed
* Completed
|-
! scope="row"| 11
|
* 11/30/2021
|
* 12/06/2021
|
* Continue fine-tune user experience
* Integrate hardware
|
* Completed
* Completed
|-
! scope="row"| 12
|
* 12/07/2021
|
* 12/16/2021
|
* Final testing
* Final demo
|
* Completed
* Completed
|-
|}

== Bill Of Materials ==
{| class="wikitable" style="margin-left: 0px; margin-right: auto;"
|-
! scope="col"| Item#
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| [https://www.embeddedartists.com/products/lpc4088-developers-kit LPC4088-32 Developer's Kit]
| 1
| $300.00
|-
! scope="row"| 2
| Microcontroller Board
| [http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems/page/sj2-board SJ2 Board]
| 1
| $50.00
|-
! scope="row"| 3
| Bluetooth Module
| [https://hub.digi.com/support/products/digi-xbee/digi-xbee-s1 Digi XBee-S1]
| 2
| N/A
|-
! scope="row"| 4
| DAC PCB
| [https://jlcpcb.com JLCPCB (Set of 5)]
| 1
| $15.00
|-
! scope="row"| 4
| Monitor
| Asus TFT Monitor
| 1
| N/A
|-
! scope="row"| 5
| Mini Breadboard
| [https://www.amazon.com/EL-CK-002-Electronic-Breadboard-Capacitor-Potentiometer/dp/B01ERP6WL4/ref=pd_sbs_147_2/132-6936347-2117569?_encoding=UTF8&pd_rd_i=B01ERP6WL4&pd_rd_r=6a783095-ec87-42ae-9f01-a5ebc9a44607&pd_rd_w=V2aU6&pd_rd_wg=11h1i&pf_rd_p=d66372fe-68a6-48a3-90ec-41d7f64212be&pf_rd_r=368WNKXFBBQY2FJQ4119&psc=1&refRID=368WNKXFBBQY2FJQ4119 ELEGOO Mini Breadboard]
| 1
| $13.99
|-
! scope="row"| 6
| Arcade buttons
| [https://www.amazon.com/EG-STARTS-Illuminated-Buttons-Raspberry/dp/B01N11BDX9?pd_rd_w=DiSpt&pf_rd_p=5bc65336-a8b0-4888-9e70-4d4ce2f6bd59&pf_rd_r=JCH2VQYXFPHND4XZ4FAM&pd_rd_r=8d6243dd-e1bd-4dae-9b19-be2e00f3085f&pd_rd_wg=K4lt9&pd_rd_i=B01N11BDX9&psc=1&ref_=pd_bap_d_rp_1_i EG STARTS LED Arcade Buttons]
| 2
| $11.99
|-
! scope="row"| 7
| VGA cable
| N/A
| 1
| N/A
|-
! scope="row"| 7
| SD card
| N/A
| 1
| N/A
|-
|}

== Design & Implementation ==

=== Hardware Design ===
We designed our PCB using Autodesk EAGLE based on the tutorial on PCB design during the class lecture. We ordered our PCB to be manufactured from JLCPCB. The PCB consists of an R-2R resistor ladder DAC for the purpose of converting the digital signal from the microcontroller to analog to display graphics on the VGA monitor. Since the VGA input for each color channel is 0-0.7V, the resistor ladder scaled the LPC4088 outputs into proper DAC values for the VGA to take in. Five bits are passed to each of the red, green, and blue channels of the VGA in addition to other signals, such as Horizontal sync (HSYNC) and Vertical sync (VSYNC).
 
[[File:pcbSchematic.png|500px|thumb|left|PCB Schematic]]
[[File:pcbLayout.png|500px|thumb|right|PCB Layout]]
[[File:pcbPic-min.png|500px|thumb|right|Manufactured PCB]]
[[File:pcbManufactured.png|500px|thumb|left|Soldered PCB]]
 

=== Hardware Interface ===

Two Digi XBee devices were used to communicate between the SJ2-Board controller and the LPC4088 board to control the paddle on the game. The XBee devices were programmed using the XTCU programming application provided by Digi International. The SJ2-Board used one XBee as a transmitted and the LPC4088 used another as a receiver. The two modules communicated using UART frames and a single byte represented any change of movement or button presses sent by the controller.

{| class="wikitable"
|-
! scope="col"| XCTU
! scope="col"| Transmitter
! scope="col"| Receiver
|-
! scope="row"| CH
| C
| C
|-
! scope="row"| ID
| 6969
| 6969
|-
! scope="row"| CE
| Coordinator
| Endpoint
|-
! scope="row"| Baud Rate
| 9600
| 9600
|-
|}
[[File:sj2Board.png|380px|thumb|center|SJ2 Board]]
[[File:CMPE146_F16_MW_XbeeModule.jpg|200px|thumb|center|XBee Device]]
 

Two LED buttons were used to interface with the controller as inputs. They were both set up using GPIO.
[[File:VGA LED Button.jpg|200px|thumb|center|LED Button]]
 
[[File:VGA_Controller_Block_Diagram.PNG|380px|thumb|center|Controller Block Diagram]]
 

==== I2S Peripheral and Stereo Audio Decoder ====

The LPC4088 is equipped with an I2S stereo encoder. This module provides a hardware-enabled solution for sending I2S-encoded audio data packets to stereo decoders, which produce the corresponding audio waveform. For this project, since 16-bit stereo audio playback was chosen, data packets transmitted on this interface were sent in the following format:

[[File:I2S_audio_data_word.png]]

Here, each word of data encodes a single packet of audio in two 16-bit sections: one signal level for the left channel, and another for the right. These 4 bytes are sent at a rate of 44100 Hz, resulting in a total data transmission of 176400 bytes a second. Rather than needing to be software-defined, this transmission is automatically handled by hardware using a 8-word deep FIFO queue that manages streaming out the audio data at a fixed rate, saving a significant amount of CPU cycles.

This encoded audio signal was sent to a dedicated external [https://www.nxp.com/docs/en/data-sheet/UDA1380.pdf stereo decoder], which output to a 3.5 mm audio jack connected to speakers in the VGA monitor.

==== LCD Peripheral ====

The LCD peripheral was configured for 16-bit color palette (5 bits of color data for each color level -- red, green, and blue -- and an unused intensity bit), at a resolution of 800 by 600 pixels, and with an active color palette size of 256 colors. Since the logic for driving an LCD display has the same concepts (horizontal blanking, vertical blanking, etc) and timing information for any given resolution was the same, we were able to directly drive a VGA display with the same signals and timing data (after the digital signal had been converted to analog).

[[File:LCD_%2B_VGA_Video_signal.png]]

The timing register configurations needed to achieve the requested video display was derived from a dedicated website for [http://tinyvga.com/vga-timing/800x600@60Hz VGA signal info].

==== External Memory ====

The game's visual assets and tracker audio file (both after being loaded from an SD card) and the game's video buffer required more memory than what is present natively in the microcontroller. To accommodate this, an external 32MiB memory module was used. This module was interfaced with using the external memory controller peripheral present on the board.

=== Software Design ===
==== Controller ====

There is an internal accelerometer on the SJ2 board that is utilized as the movement controls for the paddle movement. The y-axis value of the magnitude is polled 17 times in 17 milliseconds and is divided by 17 to get an average reading per that time frame. The maximum tilt value that can be read by the controller is set to 700 out of the total 1000 (highest possible value). The current tilt value is divided by 31 to get a total of 32 possible readings to represent 5 bits of magnitude.

The controller sends one byte of data to the console whenever there is any change in magnitude on the accelerometer or button changes. The byte contains all the information needed to control the paddle on the screen. The first two bits on the byte represent 0 if there is no button input and 1 if there is a button input. The third bit represents whether the controller is tilted left or right with 0 being left and 1 being right. The last five bits represent the controller accelerometer's magnitude.

[[File:VGA_UART_Frame.PNG]]

==== Audio ====

Sound design was provided to the project (in the form of music and sound effects) through the use of a [https://https://en.wikipedia.org/wiki/Music_tracker tracker audio library]. Source code was adopted from the [https://github.com/schismtracker/schismtracker Schism Tracker] library and altered with the following modifications:

- All UI logic was stripped from the source code

- All file handling logic was reworked to use the SD card reading functions, as the project does not support POSIX file calls

- The [https://www.libsdl.org/ Simple DirectMedia Layer] hooks were replaced with a FreeRTOS task that periodically called the waveform generation callback function, which was passed to the I2S peripheral hooked up to a stereo decoder.

=== Implementation ===

==== Ball Logic ====

The ball is controlled by the buttons and the controller accelerometer values. When the button is pressed, the ball is initially sent with a random velocity. The velocity is determined by a substate timer and is initially set as one of 6 possible values.

{| class="wikitable"
|-
! scope="col"| Substate Count
! scope="col"| X-Velocity
! scope="col"| Y-Velocity
|-
! scope="row"| 0
| 4
| -2
|-
! scope="row"| 1
| -4
| -2
|-
! scope="row"| 2
| 2
| -4
|-
! scope="row"| 3
| -2
| -4
|-
! scope="row"| 4
| 1
| -5
|-
! scope="row"| 5
| -1
| -5
|-
|}

The ball's velocity changes whenever it detects a collision with either the side of the screen, the bricks on the top of the level, or the paddle at the bottom. If colliding with the paddle, the ball's velocity will take into account the paddle's velocity and add it to its current velocity. This is done by adding the paddle's x-velocity and adjusting the y-velocity by normalizing the velocity vectors to have a magnitude of 5. This means the ball will always be moving with a magnitude of 5 in any direction after it hits the paddle. If the ball collides with anything other than the paddle, then the velocity of the ball is changed with no increase or decrease of magnitude. If the ball collides on the side of a wall or block, the x-velocity will reflect (go from negative to positive or vice-versa) and the y-velocity will remain constant. If the ball collides on the top or bottom of a block, the y-velocity will reflect and the x-velocity will remain constant. If it appears that the ball collides with the side and top or bottom of something, then the ball will check whichever face it will collide with first and then calculate the other face. Lastly, when the ball collides with the bottom of the screen, then the player experiences a "death" and the ball will reset on the paddle. If the ball collides with the top of the screen, the screen will scroll up for the user to play on the next level.

Whenever the ball collides with a brick, the brick will disappear. To accomplish this, there is a 2-D array of unsigned integers that contains the values of the bricks. If the integer is 0, then there is no block. If the integer is any value other than 0, then the block is filled and has collision logic. If the ball approaches a region that might have a block, then there will be a check within that 2-D to see if the block is filled or empty. If the block is filled, there will be the collision logic that changes the vector of the ball and sets the integer value in the 2-D array to 0 to represent that the block has been broken.

==== Paddle Movement ====
The paddle movement is controlled by the accelerometer data being wirelessly transmitted from the SJ-Two board to the LPC4088 board. The accelerometer data will map to 63 possible indexes for the paddle location. Originally, the paddle would jump directly into the corresponding index location depending on the accelerometer input. We later iterated on this process by implementing an algorithm to smoothly transition into these indexes. This was done by calculating a paddle velocity using the "current desired paddle position" and the previous paddle position. We adjusted the parameters of the algorithm iteratively to produce the smooth paddle movement.

==== Paddle Collision Logic ====
The position of the ball is calculated by adding delta_x and delta_y to the ball's current x and y position. For each frame, if the ball's next Y position crosses the paddle's Y position, we would check for a paddle collision. To do this, we used the delta_x and delta_y to get the ball's movement as a slope. We then substituted the slope, the ball's current position, and 1 pixel above the paddle's Y position into a point-slope equation (y2 - y1) = m(x2 - x1) to get the paddle's X position at the time of the ball reaching the pixel above paddle's Y position. After finding the location where the ball is expected to collide with the paddle, we then calculated the ball's collision magnitude and the ball's next location's magnitude. Using a ratio of these two magnitudes, we have a frame time fraction of when the ball possibly collided with the paddle. Then, by using the paddle's current position and paddle velocity, we are able to find where the paddle previously was, and measure how far the paddle had moved during that time frame. By multiplying the time frame from the ball's collision magnitude to the paddle's change in X position, we are able to find whether the paddle was under the ball at the time of the ball reaching the pixel above the paddle's Y position.

== Testing & Technical Challenges ==

=== VGA Interfacing ===
A DAC resistor ladder was designed and a corresponding custom PCB was acquired to interface the 3.3V digital LCD controller signal with the analog VGA input signal. The VGA cable takes between 0V (completely dark) and 0.7V (maximum brightness) for color pin inputs. The DAC resistor ladder was designed and implemented on a breadboard to verify the design. Validating the design on a breadboard gave us a clear vision during PCB design and planning.

=== VGA Write Timings ===
There were initial tests that were required to be done to see if using a VGA monitor as an output was a viable option for this project. We had to test memory write timings to see if they were fast enough to match the refresh rate of the monitor. Those tests would decide how much of the screen buffer we could write to during the blanking periods during the VGA refresh. This was a good learning experience for testing hardware capabilities and should be done whenever introducing any new aspects to a project.

=== Ball Collision Logic ===
The initial version of the ball collision used pixel values to see if the ball was entering the brick. The problem was that this logic did not account for the case where the ball enters the brick diagonally and the logic for that version of the collision had to be scrapped. The later iteration of the logic was changed to checking whether the ball was potentially crossing into an area that had a brick. This was done by separating the checks into a grid that represented the locations of the blocks. In this case, there was an initial check to see if the ball was crossing a boundary where there might be a new brick. If the ball is crossing the x-boundary and not the y-boundary, then there would be a collision check to see if it collided with a block. If it is crossing the y-boundary and not the x-boundary, once again then there would be a collision check to see if it collided with a block. If it is crossing both the x-boundary and y-boundary, then there would be a check to see which boundary the ball would cross first. This new logic allowed for us to account for the case where the block enters the block diagonally. This taught us that sometimes it is beneficial to scrap a previous iteration and use a new approach to solving a problem because it may lead to better results.

=== Paddle Movement ===
One issue we discovered was the paddle would slow down as it approached the walls. This was due to the difference in "current desired paddle position" and the previous paddle position becoming smaller as the walls were approached, leading to a smaller paddle velocity. To account for this, whenever the accelerometer sent data that mapped to the minimum or maximum paddle index, we would increase the distance of the "current desired paddle position" so the paddle velocity would not slow the paddle down as it approached the walls.

=== Paddle Collision Logic ===
During final integration and testing, one of the biggest issues we encountered was a bug where the ball would not detect a collision with the paddle. The ball would phase right through the paddle instead of bouncing off of it. One of the great ideas that we thought of was to change the color of the ball during each check to see if the checks would go through. This debug check allowed us to verify that section of code that checked for paddle collisions was being skipped even though the ball's next Y position had crossed the paddle's Y position. By having this debugging method, we were able to root cause the issue.

== Conclusion ==
Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

=== Project Video ===
[https://www.youtube.com/watch?v=ipngAGlN86c SJSU CMPE 244 FALL 2021: Embedded Software Project Gameplay]

=== Project Source Code ===
* [https://gitlab.com/fayek.wahhab/sjtwo-c/-/tree/feature/i2s_dma2 Treasure Dive Baseline for Demo Source Code]
* [https://gitlab.com/hobriandh/sjtwo-c/-/tree/final_controller Controller for Demo Source Code]

== References ==
=== Acknowledgement ===
We would like to acknowledge and thank our instructor Preetpal Kang for sharing his knowledge in embedded software development using FreeRTOS.

=== Appendix ===
*[https://gitlab.com/sjtwo-c-dev/sjtwo-c/-/blob/master/LPC408X_7X%20User%20Manual.pdf LPC408x/407x Datasheet]
*[https://www.freertos.org/Documentation/RTOS_book.html FreeRTOS Documentation]
*[http://elm-chan.org/fsw/ff/00index_e.html FatFs - Generic FAT Filesystem Module]
*[https://www.digi.com/resources/library/data-sheets/ds_xbeemultipointmodules Xbee Datasheet]
*[http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems Embedded Drivers & Real Time Operating Systems Lessons]

File:Cmpe-244-billy-pic.png

2021-12-18T12:14:59Z

Proj user1:

F21: Treasure Dive

2021-12-18T12:05:43Z

Proj user1: /* Objectives & Introduction */

== '''Treasure Dive''' ==
[[File:mermaid.png|thumb|500px|right|Game Screen Capture]]
[[File:Cmpe-244-controller-pic.png|thumb|600px|right|Game Controller]]

== Abstract ==
Treasure Dive is a modern one player game inspired by the popular arcade game Breakout, which was released by Atari in 1976. The ancients rambled about an unconfirmed tale of a shipwreck that saw no survivors. As an adventurous and skilled free diver, you are seeking treasure buried deep in an ocean abyss. You dove down all the way to the seafloor and found the buried treasure but need to make it back to the surface to see another day! The player uses a wireless controller to control an on-screen paddle to break bricks on the top of the screen. When enough bricks are broken, a passage is revealed, allowing you to get closer to the surface.

== Objectives & Introduction ==
The objective of this project is to interface an LPC4088 to a VGA monitor by leveraging the LCD controller and use an accelerometer as the sensor for the player to control the paddle.
Two Zigbees RF modules will be integrated for wireless communication between the controller and LPC4088 board via UART protocol.
Digital audio output will be enabled by developing an I2S driver and porting an audio tracker library.
The General Purpose Direct Memory Access (GPDMA) controller memory-to-memory function will be utilized to decrease memory write times when updating the video buffer.
The DMA controller's memory-to-peripheral function will help reduce the frequent I2S interrupts when loading digital audio contents.
An SD card peripheral driver to facilitate memory exchange between an SD card will also be implemented.

=== Team Members & Responsibilities ===
* [[File:Cmpe-244-brian-pic.png ‎]]
* Brian Ho
** Wiki project schedule planning
** Interfacing the accelerometer
** Digital to Analog Conversion resistor ladder design used to interface digital LCD controller signal with analog VGA port signal
** PCB planning and acquisition
** Procurement of controller hardware
** Wireless communication via Zigbees RF modules and UART protocol
** Wireless controller logic and integration
** Ball movement and wall/brick collision logic
** Hardware/software development and integration
** Final testing and integration
* [https://www.linkedin.com/in/william-lai-525681139 Billy Lai]
** Wiki project schedule planning
** Increased CPU clock rate from 96 MHz to 120 MHz
** Initial memory write timing and measurements to estimate memory write capacity
** DMA driver memory-to-memory implementation for transferring graphics data to video buffer
** I2S with DMA memory-to-peripheral implementation to reduce memory-write interrupts when I2S Tx FIFO level is low
** Paddle movement and paddle collision logic, testing and integration
** Software development and integration
** Final testing and integration
* [[File:Cmpe-244-jasdip-pic.png]]
* Jasdip Sekhon
** Wiki project schedule planning
** PCB design and planning
** File loading method algorithm for storing loaded files in memory
** Audio track and audio effects selection for gameplay audio
** Palette manipulation for gameplay
** Software development and integration
** Final testing and integration
* [[File:Cmpe-244-isaac-pic.png ]]
* Isaac Wahhab
** Wiki project schedule planning
** DAC resistor ladder design and integration
** Hardware procurement (LPC4088 board, Zigbees, VGA monitor, etc)
** VGA driving through LCD controller
** Audio tracker library port
** Digital audio output using I2S
** Palette scrolling feature implementation
** Graphics rendering
** Vertical scrolling implementation for the video buffer
** Sprite testing and implementation for the ball
** SD card peripheral driver for reading assets from SD card
** Art assets acquisition
** Hardware/software development and integration
** Gameplay logic design
** Gameplay audio and level design
** Final testing and integration

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
*09/21/2021
|
*09/27/2021
|
* Read previous projects, gather information and discuss among the group members.
* Initial ordering of parts (LPC 4088 board, 64x64 LED screen)
|
* Completed
* Completed
|-
! scope="row"| 2
|
*09/28/2021
|
*10/04/2021
|
* Submit project proposals
|
* Completed
|-
! scope="row"| 3
|
*10/05/2021
|
*10/11/2021
|
* Test if board can drive VGA
* Test varying clock rates
* Hardware cursor on VGA
* Rough display driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 4
|
* 10/12/2021
|
* 10/18/2021
|
* DAC resistor ladder
* RGB channels
* Palette shifting
* Measure memory buffer writing timing
* Wiki schedule planning
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
*10/19/2021
|
*10/25/2021
|
* Calculate upper-bound for VGA refresh timings
* Create GitLab repository for project
* Joystick functionality
* SD card driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
*10/26/2021
|
*11/01/2021
|
* Audio peripheral
* Wireless communication using Zigbee module
* PCB planning
* Accelerometer driver
* Continue SD card driver
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/02/2021

|
* 11/08/2021
|
* Continue audio peripheral
* Continue wireless communication using Zigbee module
* Continue accelerometer driver
* PCB finalizing and ordering
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 11/09/2021
|
* 11/15/2021
|
* Convert PNG data to graphics format
* Game background functionality
* Sprite functionality
* User interface
* Integrate components
* Initial testing
|
* Completed
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 11/16/2021
|
* 11/22/2021
|
* Continue everything from last week
* Game physics (wall collision and AI movement)
* Measurements for wooden case
* Art assets (audio and visual)
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 10
|
*11/23/2021
|
*11/29/2021
|

* Fine-tune user experience
* Make case
|
* Completed
* Completed
|-
! scope="row"| 11
|
* 11/30/2021
|
* 12/06/2021
|
* Continue fine-tune user experience
* Integrate hardware
|
* Completed
* Completed
|-
! scope="row"| 12
|
* 12/07/2021
|
* 12/16/2021
|
* Final testing
* Final demo
|
* Completed
* Completed
|-
|}

== Bill Of Materials ==
{| class="wikitable" style="margin-left: 0px; margin-right: auto;"
|-
! scope="col"| Item#
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| [https://www.embeddedartists.com/products/lpc4088-developers-kit LPC4088-32 Developer's Kit]
| 1
| $300.00
|-
! scope="row"| 2
| Microcontroller Board
| [http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems/page/sj2-board SJ2 Board]
| 1
| $50.00
|-
! scope="row"| 3
| Bluetooth Module
| [https://hub.digi.com/support/products/digi-xbee/digi-xbee-s1 Digi XBee-S1]
| 2
| N/A
|-
! scope="row"| 4
| DAC PCB
| [https://jlcpcb.com JLCPCB (Set of 5)]
| 1
| $15.00
|-
! scope="row"| 4
| Monitor
| Asus TFT Monitor
| 1
| N/A
|-
! scope="row"| 5
| Mini Breadboard
| [https://www.amazon.com/EL-CK-002-Electronic-Breadboard-Capacitor-Potentiometer/dp/B01ERP6WL4/ref=pd_sbs_147_2/132-6936347-2117569?_encoding=UTF8&pd_rd_i=B01ERP6WL4&pd_rd_r=6a783095-ec87-42ae-9f01-a5ebc9a44607&pd_rd_w=V2aU6&pd_rd_wg=11h1i&pf_rd_p=d66372fe-68a6-48a3-90ec-41d7f64212be&pf_rd_r=368WNKXFBBQY2FJQ4119&psc=1&refRID=368WNKXFBBQY2FJQ4119 ELEGOO Mini Breadboard]
| 1
| $13.99
|-
! scope="row"| 6
| Arcade buttons
| [https://www.amazon.com/EG-STARTS-Illuminated-Buttons-Raspberry/dp/B01N11BDX9?pd_rd_w=DiSpt&pf_rd_p=5bc65336-a8b0-4888-9e70-4d4ce2f6bd59&pf_rd_r=JCH2VQYXFPHND4XZ4FAM&pd_rd_r=8d6243dd-e1bd-4dae-9b19-be2e00f3085f&pd_rd_wg=K4lt9&pd_rd_i=B01N11BDX9&psc=1&ref_=pd_bap_d_rp_1_i EG STARTS LED Arcade Buttons]
| 2
| $11.99
|-
! scope="row"| 7
| VGA cable
| N/A
| 1
| N/A
|-
! scope="row"| 7
| SD card
| N/A
| 1
| N/A
|-
|}

== Design & Implementation ==

=== Hardware Design ===
We designed our PCB using Autodesk EAGLE based on the tutorial on PCB design during the class lecture. We ordered our PCB to be manufactured from JLCPCB. The PCB consists of an R-2R resistor ladder DAC for the purpose of converting the digital signal from the microcontroller to analog to display graphics on the VGA monitor. Since the VGA input for each color channel is 0-0.7V, the resistor ladder scaled the LPC4088 outputs into proper DAC values for the VGA to take in. Five bits are passed to each of the red, green, and blue channels of the VGA in addition to other signals, such as Horizontal sync (HSYNC) and Vertical sync (VSYNC).
 
[[File:pcbSchematic.png|500px|thumb|left|PCB Schematic]]
[[File:pcbLayout.png|500px|thumb|right|PCB Layout]]
[[File:pcbPic-min.png|500px|thumb|right|Manufactured PCB]]
[[File:pcbManufactured.png|500px|thumb|left|Soldered PCB]]
 

=== Hardware Interface ===

Two Digi XBee devices were used to communicate between the SJ2-Board controller and the LPC4088 board to control the paddle on the game. The XBee devices were programmed using the XTCU programming application provided by Digi International. The SJ2-Board used one XBee as a transmitted and the LPC4088 used another as a receiver. The two modules communicated using UART frames and a single byte represented any change of movement or button presses sent by the controller.

{| class="wikitable"
|-
! scope="col"| XCTU
! scope="col"| Transmitter
! scope="col"| Receiver
|-
! scope="row"| CH
| C
| C
|-
! scope="row"| ID
| 6969
| 6969
|-
! scope="row"| CE
| Coordinator
| Endpoint
|-
! scope="row"| Baud Rate
| 9600
| 9600
|-
|}
[[File:sj2Board.png|380px|thumb|center|SJ2 Board]]
[[File:CMPE146_F16_MW_XbeeModule.jpg|200px|thumb|center|XBee Device]]
 

Two LED buttons were used to interface with the controller as inputs. They were both set up using GPIO.
[[File:VGA LED Button.jpg|200px|thumb|center|LED Button]]
 
[[File:VGA_Controller_Block_Diagram.PNG|380px|thumb|center|Controller Block Diagram]]
 

==== I2S Peripheral and Stereo Audio Decoder ====

The LPC4088 is equipped with an I2S stereo encoder. This module provides a hardware-enabled solution for sending I2S-encoded audio data packets to stereo decoders, which produce the corresponding audio waveform. For this project, since 16-bit stereo audio playback was chosen, data packets transmitted on this interface were sent in the following format:

[[File:I2S_audio_data_word.png]]

Here, each word of data encodes a single packet of audio in two 16-bit sections: one signal level for the left channel, and another for the right. These 4 bytes are sent at a rate of 44100 Hz, resulting in a total data transmission of 176400 bytes a second. Rather than needing to be software-defined, this transmission is automatically handled by hardware using a 8-word deep FIFO queue that manages streaming out the audio data at a fixed rate, saving a significant amount of CPU cycles.

This encoded audio signal was sent to a dedicated external [https://www.nxp.com/docs/en/data-sheet/UDA1380.pdf stereo decoder], which output to a 3.5 mm audio jack connected to speakers in the VGA monitor.

==== LCD Peripheral ====

The LCD peripheral was configured for 16-bit color palette (5 bits of color data for each color level -- red, green, and blue -- and an unused intensity bit), at a resolution of 800 by 600 pixels, and with an active color palette size of 256 colors. Since the logic for driving an LCD display has the same concepts (horizontal blanking, vertical blanking, etc) and timing information for any given resolution was the same, we were able to directly drive a VGA display with the same signals and timing data (after the digital signal had been converted to analog).

[[File:LCD_%2B_VGA_Video_signal.png]]

The timing register configurations needed to achieve the requested video display was derived from a dedicated website for [http://tinyvga.com/vga-timing/800x600@60Hz VGA signal info].

==== External Memory ====

The game's visual assets and tracker audio file (both after being loaded from an SD card) and the game's video buffer required more memory than what is present natively in the microcontroller. To accommodate this, an external 32MiB memory module was used. This module was interfaced with using the external memory controller peripheral present on the board.

=== Software Design ===
==== Controller ====

There is an internal accelerometer on the SJ2 board that is utilized as the movement controls for the paddle movement. The y-axis value of the magnitude is polled 17 times in 17 milliseconds and is divided by 17 to get an average reading per that time frame. The maximum tilt value that can be read by the controller is set to 700 out of the total 1000 (highest possible value). The current tilt value is divided by 31 to get a total of 32 possible readings to represent 5 bits of magnitude.

The controller sends one byte of data to the console whenever there is any change in magnitude on the accelerometer or button changes. The byte contains all the information needed to control the paddle on the screen. The first two bits on the byte represent 0 if there is no button input and 1 if there is a button input. The third bit represents whether the controller is tilted left or right with 0 being left and 1 being right. The last five bits represent the controller accelerometer's magnitude.

[[File:VGA_UART_Frame.PNG]]

==== Audio ====

Sound design was provided to the project (in the form of music and sound effects) through the use of a [https://https://en.wikipedia.org/wiki/Music_tracker tracker audio library]. Source code was adopted from the [https://github.com/schismtracker/schismtracker Schism Tracker] library and altered with the following modifications:

- All UI logic was stripped from the source code

- All file handling logic was reworked to use the SD card reading functions, as the project does not support POSIX file calls

- The [https://www.libsdl.org/ Simple DirectMedia Layer] hooks were replaced with a FreeRTOS task that periodically called the waveform generation callback function, which was passed to the I2S peripheral hooked up to a stereo decoder.

=== Implementation ===

==== Ball Logic ====

The ball is controlled by the buttons and the controller accelerometer values. When the button is pressed, the ball is initially sent with a random velocity. The velocity is determined by a substate timer and is initially set as one of 6 possible values.

{| class="wikitable"
|-
! scope="col"| Substate Count
! scope="col"| X-Velocity
! scope="col"| Y-Velocity
|-
! scope="row"| 0
| 4
| -2
|-
! scope="row"| 1
| -4
| -2
|-
! scope="row"| 2
| 2
| -4
|-
! scope="row"| 3
| -2
| -4
|-
! scope="row"| 4
| 1
| -5
|-
! scope="row"| 5
| -1
| -5
|-
|}

The ball's velocity changes whenever it detects a collision with either the side of the screen, the bricks on the top of the level, or the paddle at the bottom. If colliding with the paddle, the ball's velocity will take into account the paddle's velocity and add it to its current velocity. This is done by adding the paddle's x-velocity and adjusting the y-velocity by normalizing the velocity vectors to have a magnitude of 5. This means the ball will always be moving with a magnitude of 5 in any direction after it hits the paddle. If the ball collides with anything other than the paddle, then the velocity of the ball is changed with no increase or decrease of magnitude. If the ball collides on the side of a wall or block, the x-velocity will reflect (go from negative to positive or vice-versa) and the y-velocity will remain constant. If the ball collides on the top or bottom of a block, the y-velocity will reflect and the x-velocity will remain constant. If it appears that the ball collides with the side and top or bottom of something, then the ball will check whichever face it will collide with first and then calculate the other face. Lastly, when the ball collides with the bottom of the screen, then the player experiences a "death" and the ball will reset on the paddle. If the ball collides with the top of the screen, the screen will scroll up for the user to play on the next level.

Whenever the ball collides with a brick, the brick will disappear. To accomplish this, there is a 2-D array of unsigned integers that contains the values of the bricks. If the integer is 0, then there is no block. If the integer is any value other than 0, then the block is filled and has collision logic. If the ball approaches a region that might have a block, then there will be a check within that 2-D to see if the block is filled or empty. If the block is filled, there will be the collision logic that changes the vector of the ball and sets the integer value in the 2-D array to 0 to represent that the block has been broken.

==== Paddle Movement ====
The paddle movement is controlled by the accelerometer data being wirelessly transmitted from the SJ-Two board to the LPC4088 board. The accelerometer data will map to 63 possible indexes for the paddle location. Originally, the paddle would jump directly into the corresponding index location depending on the accelerometer input. We later iterated on this process by implementing an algorithm to smoothly transition into these indexes. This was done by calculating a paddle velocity using the "current desired paddle position" and the previous paddle position. We adjusted the parameters of the algorithm iteratively to produce the smooth paddle movement.

==== Paddle Collision Logic ====
The position of the ball is calculated by adding delta_x and delta_y to the ball's current x and y position. For each frame, if the ball's next Y position crosses the paddle's Y position, we would check for a paddle collision. To do this, we used the delta_x and delta_y to get the ball's movement as a slope. We then substituted the slope, the ball's current position, and 1 pixel above the paddle's Y position into a point-slope equation (y2 - y1) = m(x2 - x1) to get the paddle's X position at the time of the ball reaching the pixel above paddle's Y position. After finding the location where the ball is expected to collide with the paddle, we then calculated the ball's collision magnitude and the ball's next location's magnitude. Using a ratio of these two magnitudes, we have a frame time fraction of when the ball possibly collided with the paddle. Then, by using the paddle's current position and paddle velocity, we are able to find where the paddle previously was, and measure how far the paddle had moved during that time frame. By multiplying the time frame from the ball's collision magnitude to the paddle's change in X position, we are able to find whether the paddle was under the ball at the time of the ball reaching the pixel above the paddle's Y position.

== Testing & Technical Challenges ==

=== VGA Interfacing ===
A DAC resistor ladder was designed and a corresponding custom PCB was acquired to interface the 3.3V digital LCD controller signal with the analog VGA input signal. The VGA cable takes between 0V (completely dark) and 0.7V (maximum brightness) for color pin inputs. The DAC resistor ladder was designed and implemented on a breadboard to verify the design. Validating the design on a breadboard gave us a clear vision during PCB design and planning.

=== VGA Write Timings ===
There were initial tests that were required to be done to see if using a VGA monitor as an output was a viable option for this project. We had to test memory write timings to see if they were fast enough to match the refresh rate of the monitor. Those tests would decide how much of the screen buffer we could write to during the blanking periods during the VGA refresh. This was a good learning experience for testing hardware capabilities and should be done whenever introducing any new aspects to a project.

=== Ball Collision Logic ===
The initial version of the ball collision used pixel values to see if the ball was entering the brick. The problem was that this logic did not account for the case where the ball enters the brick diagonally and the logic for that version of the collision had to be scrapped. The later iteration of the logic was changed to checking whether the ball was potentially crossing into an area that had a brick. This was done by separating the checks into a grid that represented the locations of the blocks. In this case, there was an initial check to see if the ball was crossing a boundary where there might be a new brick. If the ball is crossing the x-boundary and not the y-boundary, then there would be a collision check to see if it collided with a block. If it is crossing the y-boundary and not the x-boundary, once again then there would be a collision check to see if it collided with a block. If it is crossing both the x-boundary and y-boundary, then there would be a check to see which boundary the ball would cross first. This new logic allowed for us to account for the case where the block enters the block diagonally. This taught us that sometimes it is beneficial to scrap a previous iteration and use a new approach to solving a problem because it may lead to better results.

=== Paddle Movement ===
One issue we discovered was the paddle would slow down as it approached the walls. This was due to the difference in "current desired paddle position" and the previous paddle position becoming smaller as the walls were approached, leading to a smaller paddle velocity. To account for this, whenever the accelerometer sent data that mapped to the minimum or maximum paddle index, we would increase the distance of the "current desired paddle position" so the paddle velocity would not slow the paddle down as it approached the walls.

=== Paddle Collision Logic ===
During final integration and testing, one of the biggest issues we encountered was a bug where the ball would not detect a collision with the paddle. The ball would phase right through the paddle instead of bouncing off of it. One of the great ideas that we thought of was to change the color of the ball during each check to see if the checks would go through. This debug check allowed us to verify that section of code that checked for paddle collisions was being skipped even though the ball's next Y position had crossed the paddle's Y position. By having this debugging method, we were able to root cause the issue.

== Conclusion ==
Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

=== Project Video ===
[https://www.youtube.com/watch?v=ipngAGlN86c SJSU CMPE 244 FALL 2021: Embedded Software Project Gameplay]

=== Project Source Code ===
* [https://gitlab.com/fayek.wahhab/sjtwo-c/-/tree/feature/i2s_dma2 Treasure Dive Baseline for Demo Source Code]
* [https://gitlab.com/hobriandh/sjtwo-c/-/tree/final_controller Controller for Demo Source Code]

== References ==
=== Acknowledgement ===
We would like to acknowledge and thank our instructor Preetpal Kang for sharing his knowledge in embedded software development using FreeRTOS.

=== Appendix ===
*[https://gitlab.com/sjtwo-c-dev/sjtwo-c/-/blob/master/LPC408X_7X%20User%20Manual.pdf LPC408x/407x Datasheet]
*[https://www.freertos.org/Documentation/RTOS_book.html FreeRTOS Documentation]
*[http://elm-chan.org/fsw/ff/00index_e.html FatFs - Generic FAT Filesystem Module]
*[https://www.digi.com/resources/library/data-sheets/ds_xbeemultipointmodules Xbee Datasheet]
*[http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems Embedded Drivers & Real Time Operating Systems Lessons]

File:Cmpe-244-jasdip-pic.png

2021-12-18T12:05:20Z

Proj user1:

F21: Treasure Dive

2021-12-18T12:02:35Z

Proj user1: /* Objectives & Introduction */

== '''Treasure Dive''' ==
[[File:mermaid.png|thumb|500px|right|Game Screen Capture]]
[[File:Cmpe-244-controller-pic.png|thumb|600px|right|Game Controller]]

== Abstract ==
Treasure Dive is a modern one player game inspired by the popular arcade game Breakout, which was released by Atari in 1976. The ancients rambled about an unconfirmed tale of a shipwreck that saw no survivors. As an adventurous and skilled free diver, you are seeking treasure buried deep in an ocean abyss. You dove down all the way to the seafloor and found the buried treasure but need to make it back to the surface to see another day! The player uses a wireless controller to control an on-screen paddle to break bricks on the top of the screen. When enough bricks are broken, a passage is revealed, allowing you to get closer to the surface.

== Objectives & Introduction ==
The objective of this project is to interface an LPC4088 to a VGA monitor by leveraging the LCD controller and use an accelerometer as the sensor for the player to control the paddle.
Two Zigbees RF modules will be integrated for wireless communication between the controller and LPC4088 board via UART protocol.
Digital audio output will be enabled by developing an I2S driver and porting an audio tracker library.
The General Purpose Direct Memory Access (GPDMA) controller memory-to-memory function will be utilized to decrease memory write times when updating the video buffer.
The DMA controller's memory-to-peripheral function will help reduce the frequent I2S interrupts when loading digital audio contents.
An SD card peripheral driver to facilitate memory exchange between an SD card will also be implemented.

=== Team Members & Responsibilities ===
* [[File:Cmpe-244-brian-pic.png ‎]]
* Brian Ho
** Wiki project schedule planning
** Interfacing the accelerometer
** Digital to Analog Conversion resistor ladder design used to interface digital LCD controller signal with analog VGA port signal
** PCB planning and acquisition
** Procurement of controller hardware
** Wireless communication via Zigbees RF modules and UART protocol
** Wireless controller logic and integration
** Ball movement and wall/brick collision logic
** Hardware/software development and integration
** Final testing and integration
* [https://www.linkedin.com/in/william-lai-525681139 Billy Lai]
** Wiki project schedule planning
** Increased CPU clock rate from 96 MHz to 120 MHz
** Initial memory write timing and measurements to estimate memory write capacity
** DMA driver memory-to-memory implementation for transferring graphics data to video buffer
** I2S with DMA memory-to-peripheral implementation to reduce memory-write interrupts when I2S Tx FIFO level is low
** Paddle movement and paddle collision logic, testing and integration
** Software development and integration
** Final testing and integration
* Jasdip Sekhon
** Wiki project schedule planning
** PCB design and planning
** File loading method algorithm for storing loaded files in memory
** Audio track and audio effects selection for gameplay audio
** Palette manipulation for gameplay
** Software development and integration
** Final testing and integration
* [[File:Cmpe-244-isaac-pic.png ]]
* Isaac Wahhab
** Wiki project schedule planning
** DAC resistor ladder design and integration
** Hardware procurement (LPC4088 board, Zigbees, VGA monitor, etc)
** VGA driving through LCD controller
** Audio tracker library port
** Digital audio output using I2S
** Palette scrolling feature implementation
** Graphics rendering
** Vertical scrolling implementation for the video buffer
** Sprite testing and implementation for the ball
** SD card peripheral driver for reading assets from SD card
** Art assets acquisition
** Hardware/software development and integration
** Gameplay logic design
** Gameplay audio and level design
** Final testing and integration

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
*09/21/2021
|
*09/27/2021
|
* Read previous projects, gather information and discuss among the group members.
* Initial ordering of parts (LPC 4088 board, 64x64 LED screen)
|
* Completed
* Completed
|-
! scope="row"| 2
|
*09/28/2021
|
*10/04/2021
|
* Submit project proposals
|
* Completed
|-
! scope="row"| 3
|
*10/05/2021
|
*10/11/2021
|
* Test if board can drive VGA
* Test varying clock rates
* Hardware cursor on VGA
* Rough display driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 4
|
* 10/12/2021
|
* 10/18/2021
|
* DAC resistor ladder
* RGB channels
* Palette shifting
* Measure memory buffer writing timing
* Wiki schedule planning
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
*10/19/2021
|
*10/25/2021
|
* Calculate upper-bound for VGA refresh timings
* Create GitLab repository for project
* Joystick functionality
* SD card driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
*10/26/2021
|
*11/01/2021
|
* Audio peripheral
* Wireless communication using Zigbee module
* PCB planning
* Accelerometer driver
* Continue SD card driver
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/02/2021

|
* 11/08/2021
|
* Continue audio peripheral
* Continue wireless communication using Zigbee module
* Continue accelerometer driver
* PCB finalizing and ordering
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 11/09/2021
|
* 11/15/2021
|
* Convert PNG data to graphics format
* Game background functionality
* Sprite functionality
* User interface
* Integrate components
* Initial testing
|
* Completed
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 11/16/2021
|
* 11/22/2021
|
* Continue everything from last week
* Game physics (wall collision and AI movement)
* Measurements for wooden case
* Art assets (audio and visual)
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 10
|
*11/23/2021
|
*11/29/2021
|

* Fine-tune user experience
* Make case
|
* Completed
* Completed
|-
! scope="row"| 11
|
* 11/30/2021
|
* 12/06/2021
|
* Continue fine-tune user experience
* Integrate hardware
|
* Completed
* Completed
|-
! scope="row"| 12
|
* 12/07/2021
|
* 12/16/2021
|
* Final testing
* Final demo
|
* Completed
* Completed
|-
|}

== Bill Of Materials ==
{| class="wikitable" style="margin-left: 0px; margin-right: auto;"
|-
! scope="col"| Item#
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| [https://www.embeddedartists.com/products/lpc4088-developers-kit LPC4088-32 Developer's Kit]
| 1
| $300.00
|-
! scope="row"| 2
| Microcontroller Board
| [http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems/page/sj2-board SJ2 Board]
| 1
| $50.00
|-
! scope="row"| 3
| Bluetooth Module
| [https://hub.digi.com/support/products/digi-xbee/digi-xbee-s1 Digi XBee-S1]
| 2
| N/A
|-
! scope="row"| 4
| DAC PCB
| [https://jlcpcb.com JLCPCB (Set of 5)]
| 1
| $15.00
|-
! scope="row"| 4
| Monitor
| Asus TFT Monitor
| 1
| N/A
|-
! scope="row"| 5
| Mini Breadboard
| [https://www.amazon.com/EL-CK-002-Electronic-Breadboard-Capacitor-Potentiometer/dp/B01ERP6WL4/ref=pd_sbs_147_2/132-6936347-2117569?_encoding=UTF8&pd_rd_i=B01ERP6WL4&pd_rd_r=6a783095-ec87-42ae-9f01-a5ebc9a44607&pd_rd_w=V2aU6&pd_rd_wg=11h1i&pf_rd_p=d66372fe-68a6-48a3-90ec-41d7f64212be&pf_rd_r=368WNKXFBBQY2FJQ4119&psc=1&refRID=368WNKXFBBQY2FJQ4119 ELEGOO Mini Breadboard]
| 1
| $13.99
|-
! scope="row"| 6
| Arcade buttons
| [https://www.amazon.com/EG-STARTS-Illuminated-Buttons-Raspberry/dp/B01N11BDX9?pd_rd_w=DiSpt&pf_rd_p=5bc65336-a8b0-4888-9e70-4d4ce2f6bd59&pf_rd_r=JCH2VQYXFPHND4XZ4FAM&pd_rd_r=8d6243dd-e1bd-4dae-9b19-be2e00f3085f&pd_rd_wg=K4lt9&pd_rd_i=B01N11BDX9&psc=1&ref_=pd_bap_d_rp_1_i EG STARTS LED Arcade Buttons]
| 2
| $11.99
|-
! scope="row"| 7
| VGA cable
| N/A
| 1
| N/A
|-
! scope="row"| 7
| SD card
| N/A
| 1
| N/A
|-
|}

== Design & Implementation ==

=== Hardware Design ===
We designed our PCB using Autodesk EAGLE based on the tutorial on PCB design during the class lecture. We ordered our PCB to be manufactured from JLCPCB. The PCB consists of an R-2R resistor ladder DAC for the purpose of converting the digital signal from the microcontroller to analog to display graphics on the VGA monitor. Since the VGA input for each color channel is 0-0.7V, the resistor ladder scaled the LPC4088 outputs into proper DAC values for the VGA to take in. Five bits are passed to each of the red, green, and blue channels of the VGA in addition to other signals, such as Horizontal sync (HSYNC) and Vertical sync (VSYNC).
 
[[File:pcbSchematic.png|500px|thumb|left|PCB Schematic]]
[[File:pcbLayout.png|500px|thumb|right|PCB Layout]]
[[File:pcbPic-min.png|500px|thumb|right|Manufactured PCB]]
[[File:pcbManufactured.png|500px|thumb|left|Soldered PCB]]
 

=== Hardware Interface ===

Two Digi XBee devices were used to communicate between the SJ2-Board controller and the LPC4088 board to control the paddle on the game. The XBee devices were programmed using the XTCU programming application provided by Digi International. The SJ2-Board used one XBee as a transmitted and the LPC4088 used another as a receiver. The two modules communicated using UART frames and a single byte represented any change of movement or button presses sent by the controller.

{| class="wikitable"
|-
! scope="col"| XCTU
! scope="col"| Transmitter
! scope="col"| Receiver
|-
! scope="row"| CH
| C
| C
|-
! scope="row"| ID
| 6969
| 6969
|-
! scope="row"| CE
| Coordinator
| Endpoint
|-
! scope="row"| Baud Rate
| 9600
| 9600
|-
|}
[[File:sj2Board.png|380px|thumb|center|SJ2 Board]]
[[File:CMPE146_F16_MW_XbeeModule.jpg|200px|thumb|center|XBee Device]]
 

Two LED buttons were used to interface with the controller as inputs. They were both set up using GPIO.
[[File:VGA LED Button.jpg|200px|thumb|center|LED Button]]
 
[[File:VGA_Controller_Block_Diagram.PNG|380px|thumb|center|Controller Block Diagram]]
 

==== I2S Peripheral and Stereo Audio Decoder ====

The LPC4088 is equipped with an I2S stereo encoder. This module provides a hardware-enabled solution for sending I2S-encoded audio data packets to stereo decoders, which produce the corresponding audio waveform. For this project, since 16-bit stereo audio playback was chosen, data packets transmitted on this interface were sent in the following format:

[[File:I2S_audio_data_word.png]]

Here, each word of data encodes a single packet of audio in two 16-bit sections: one signal level for the left channel, and another for the right. These 4 bytes are sent at a rate of 44100 Hz, resulting in a total data transmission of 176400 bytes a second. Rather than needing to be software-defined, this transmission is automatically handled by hardware using a 8-word deep FIFO queue that manages streaming out the audio data at a fixed rate, saving a significant amount of CPU cycles.

This encoded audio signal was sent to a dedicated external [https://www.nxp.com/docs/en/data-sheet/UDA1380.pdf stereo decoder], which output to a 3.5 mm audio jack connected to speakers in the VGA monitor.

==== LCD Peripheral ====

The LCD peripheral was configured for 16-bit color palette (5 bits of color data for each color level -- red, green, and blue -- and an unused intensity bit), at a resolution of 800 by 600 pixels, and with an active color palette size of 256 colors. Since the logic for driving an LCD display has the same concepts (horizontal blanking, vertical blanking, etc) and timing information for any given resolution was the same, we were able to directly drive a VGA display with the same signals and timing data (after the digital signal had been converted to analog).

[[File:LCD_%2B_VGA_Video_signal.png]]

The timing register configurations needed to achieve the requested video display was derived from a dedicated website for [http://tinyvga.com/vga-timing/800x600@60Hz VGA signal info].

==== External Memory ====

The game's visual assets and tracker audio file (both after being loaded from an SD card) and the game's video buffer required more memory than what is present natively in the microcontroller. To accommodate this, an external 32MiB memory module was used. This module was interfaced with using the external memory controller peripheral present on the board.

=== Software Design ===
==== Controller ====

There is an internal accelerometer on the SJ2 board that is utilized as the movement controls for the paddle movement. The y-axis value of the magnitude is polled 17 times in 17 milliseconds and is divided by 17 to get an average reading per that time frame. The maximum tilt value that can be read by the controller is set to 700 out of the total 1000 (highest possible value). The current tilt value is divided by 31 to get a total of 32 possible readings to represent 5 bits of magnitude.

The controller sends one byte of data to the console whenever there is any change in magnitude on the accelerometer or button changes. The byte contains all the information needed to control the paddle on the screen. The first two bits on the byte represent 0 if there is no button input and 1 if there is a button input. The third bit represents whether the controller is tilted left or right with 0 being left and 1 being right. The last five bits represent the controller accelerometer's magnitude.

[[File:VGA_UART_Frame.PNG]]

==== Audio ====

Sound design was provided to the project (in the form of music and sound effects) through the use of a [https://https://en.wikipedia.org/wiki/Music_tracker tracker audio library]. Source code was adopted from the [https://github.com/schismtracker/schismtracker Schism Tracker] library and altered with the following modifications:

- All UI logic was stripped from the source code

- All file handling logic was reworked to use the SD card reading functions, as the project does not support POSIX file calls

- The [https://www.libsdl.org/ Simple DirectMedia Layer] hooks were replaced with a FreeRTOS task that periodically called the waveform generation callback function, which was passed to the I2S peripheral hooked up to a stereo decoder.

=== Implementation ===

==== Ball Logic ====

The ball is controlled by the buttons and the controller accelerometer values. When the button is pressed, the ball is initially sent with a random velocity. The velocity is determined by a substate timer and is initially set as one of 6 possible values.

{| class="wikitable"
|-
! scope="col"| Substate Count
! scope="col"| X-Velocity
! scope="col"| Y-Velocity
|-
! scope="row"| 0
| 4
| -2
|-
! scope="row"| 1
| -4
| -2
|-
! scope="row"| 2
| 2
| -4
|-
! scope="row"| 3
| -2
| -4
|-
! scope="row"| 4
| 1
| -5
|-
! scope="row"| 5
| -1
| -5
|-
|}

The ball's velocity changes whenever it detects a collision with either the side of the screen, the bricks on the top of the level, or the paddle at the bottom. If colliding with the paddle, the ball's velocity will take into account the paddle's velocity and add it to its current velocity. This is done by adding the paddle's x-velocity and adjusting the y-velocity by normalizing the velocity vectors to have a magnitude of 5. This means the ball will always be moving with a magnitude of 5 in any direction after it hits the paddle. If the ball collides with anything other than the paddle, then the velocity of the ball is changed with no increase or decrease of magnitude. If the ball collides on the side of a wall or block, the x-velocity will reflect (go from negative to positive or vice-versa) and the y-velocity will remain constant. If the ball collides on the top or bottom of a block, the y-velocity will reflect and the x-velocity will remain constant. If it appears that the ball collides with the side and top or bottom of something, then the ball will check whichever face it will collide with first and then calculate the other face. Lastly, when the ball collides with the bottom of the screen, then the player experiences a "death" and the ball will reset on the paddle. If the ball collides with the top of the screen, the screen will scroll up for the user to play on the next level.

Whenever the ball collides with a brick, the brick will disappear. To accomplish this, there is a 2-D array of unsigned integers that contains the values of the bricks. If the integer is 0, then there is no block. If the integer is any value other than 0, then the block is filled and has collision logic. If the ball approaches a region that might have a block, then there will be a check within that 2-D to see if the block is filled or empty. If the block is filled, there will be the collision logic that changes the vector of the ball and sets the integer value in the 2-D array to 0 to represent that the block has been broken.

==== Paddle Movement ====
The paddle movement is controlled by the accelerometer data being wirelessly transmitted from the SJ-Two board to the LPC4088 board. The accelerometer data will map to 63 possible indexes for the paddle location. Originally, the paddle would jump directly into the corresponding index location depending on the accelerometer input. We later iterated on this process by implementing an algorithm to smoothly transition into these indexes. This was done by calculating a paddle velocity using the "current desired paddle position" and the previous paddle position. We adjusted the parameters of the algorithm iteratively to produce the smooth paddle movement.

==== Paddle Collision Logic ====
The position of the ball is calculated by adding delta_x and delta_y to the ball's current x and y position. For each frame, if the ball's next Y position crosses the paddle's Y position, we would check for a paddle collision. To do this, we used the delta_x and delta_y to get the ball's movement as a slope. We then substituted the slope, the ball's current position, and 1 pixel above the paddle's Y position into a point-slope equation (y2 - y1) = m(x2 - x1) to get the paddle's X position at the time of the ball reaching the pixel above paddle's Y position. After finding the location where the ball is expected to collide with the paddle, we then calculated the ball's collision magnitude and the ball's next location's magnitude. Using a ratio of these two magnitudes, we have a frame time fraction of when the ball possibly collided with the paddle. Then, by using the paddle's current position and paddle velocity, we are able to find where the paddle previously was, and measure how far the paddle had moved during that time frame. By multiplying the time frame from the ball's collision magnitude to the paddle's change in X position, we are able to find whether the paddle was under the ball at the time of the ball reaching the pixel above the paddle's Y position.

== Testing & Technical Challenges ==

=== VGA Interfacing ===
A DAC resistor ladder was designed and a corresponding custom PCB was acquired to interface the 3.3V digital LCD controller signal with the analog VGA input signal. The VGA cable takes between 0V (completely dark) and 0.7V (maximum brightness) for color pin inputs. The DAC resistor ladder was designed and implemented on a breadboard to verify the design. Validating the design on a breadboard gave us a clear vision during PCB design and planning.

=== VGA Write Timings ===
There were initial tests that were required to be done to see if using a VGA monitor as an output was a viable option for this project. We had to test memory write timings to see if they were fast enough to match the refresh rate of the monitor. Those tests would decide how much of the screen buffer we could write to during the blanking periods during the VGA refresh. This was a good learning experience for testing hardware capabilities and should be done whenever introducing any new aspects to a project.

=== Ball Collision Logic ===
The initial version of the ball collision used pixel values to see if the ball was entering the brick. The problem was that this logic did not account for the case where the ball enters the brick diagonally and the logic for that version of the collision had to be scrapped. The later iteration of the logic was changed to checking whether the ball was potentially crossing into an area that had a brick. This was done by separating the checks into a grid that represented the locations of the blocks. In this case, there was an initial check to see if the ball was crossing a boundary where there might be a new brick. If the ball is crossing the x-boundary and not the y-boundary, then there would be a collision check to see if it collided with a block. If it is crossing the y-boundary and not the x-boundary, once again then there would be a collision check to see if it collided with a block. If it is crossing both the x-boundary and y-boundary, then there would be a check to see which boundary the ball would cross first. This new logic allowed for us to account for the case where the block enters the block diagonally. This taught us that sometimes it is beneficial to scrap a previous iteration and use a new approach to solving a problem because it may lead to better results.

=== Paddle Movement ===
One issue we discovered was the paddle would slow down as it approached the walls. This was due to the difference in "current desired paddle position" and the previous paddle position becoming smaller as the walls were approached, leading to a smaller paddle velocity. To account for this, whenever the accelerometer sent data that mapped to the minimum or maximum paddle index, we would increase the distance of the "current desired paddle position" so the paddle velocity would not slow the paddle down as it approached the walls.

=== Paddle Collision Logic ===
During final integration and testing, one of the biggest issues we encountered was a bug where the ball would not detect a collision with the paddle. The ball would phase right through the paddle instead of bouncing off of it. One of the great ideas that we thought of was to change the color of the ball during each check to see if the checks would go through. This debug check allowed us to verify that section of code that checked for paddle collisions was being skipped even though the ball's next Y position had crossed the paddle's Y position. By having this debugging method, we were able to root cause the issue.

== Conclusion ==
Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

=== Project Video ===
[https://www.youtube.com/watch?v=ipngAGlN86c SJSU CMPE 244 FALL 2021: Embedded Software Project Gameplay]

=== Project Source Code ===
* [https://gitlab.com/fayek.wahhab/sjtwo-c/-/tree/feature/i2s_dma2 Treasure Dive Baseline for Demo Source Code]
* [https://gitlab.com/hobriandh/sjtwo-c/-/tree/final_controller Controller for Demo Source Code]

== References ==
=== Acknowledgement ===
We would like to acknowledge and thank our instructor Preetpal Kang for sharing his knowledge in embedded software development using FreeRTOS.

=== Appendix ===
*[https://gitlab.com/sjtwo-c-dev/sjtwo-c/-/blob/master/LPC408X_7X%20User%20Manual.pdf LPC408x/407x Datasheet]
*[https://www.freertos.org/Documentation/RTOS_book.html FreeRTOS Documentation]
*[http://elm-chan.org/fsw/ff/00index_e.html FatFs - Generic FAT Filesystem Module]
*[https://www.digi.com/resources/library/data-sheets/ds_xbeemultipointmodules Xbee Datasheet]
*[http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems Embedded Drivers & Real Time Operating Systems Lessons]

F21: Treasure Dive

2021-12-18T12:01:57Z

Proj user1: /* Objectives & Introduction */

== '''Treasure Dive''' ==
[[File:mermaid.png|thumb|500px|right|Game Screen Capture]]
[[File:Cmpe-244-controller-pic.png|thumb|600px|right|Game Controller]]

== Abstract ==
Treasure Dive is a modern one player game inspired by the popular arcade game Breakout, which was released by Atari in 1976. The ancients rambled about an unconfirmed tale of a shipwreck that saw no survivors. As an adventurous and skilled free diver, you are seeking treasure buried deep in an ocean abyss. You dove down all the way to the seafloor and found the buried treasure but need to make it back to the surface to see another day! The player uses a wireless controller to control an on-screen paddle to break bricks on the top of the screen. When enough bricks are broken, a passage is revealed, allowing you to get closer to the surface.

== Objectives & Introduction ==
The objective of this project is to interface an LPC4088 to a VGA monitor by leveraging the LCD controller and use an accelerometer as the sensor for the player to control the paddle.
Two Zigbees RF modules will be integrated for wireless communication between the controller and LPC4088 board via UART protocol.
Digital audio output will be enabled by developing an I2S driver and porting an audio tracker library.
The General Purpose Direct Memory Access (GPDMA) controller memory-to-memory function will be utilized to decrease memory write times when updating the video buffer.
The DMA controller's memory-to-peripheral function will help reduce the frequent I2S interrupts when loading digital audio contents.
An SD card peripheral driver to facilitate memory exchange between an SD card will also be implemented.

=== Team Members & Responsibilities ===
* [[File:Cmpe-244-brian-pic.png ‎]]
* Brian Ho
** Wiki project schedule planning
** Interfacing the accelerometer
** Digital to Analog Conversion resistor ladder design used to interface digital LCD controller signal with analog VGA port signal
** PCB planning and acquisition
** Procurement of controller hardware
** Wireless communication via Zigbees RF modules and UART protocol
** Wireless controller logic and integration
** Ball movement and wall/brick collision logic
** Hardware/software development and integration
** Final testing and integration
* [https://www.linkedin.com/in/william-lai-525681139 Billy Lai]
** Wiki project schedule planning
** Increased CPU clock rate from 96 MHz to 120 MHz
** Initial memory write timing and measurements to estimate memory write capacity
** DMA driver memory-to-memory implementation for transferring graphics data to video buffer
** I2S with DMA memory-to-peripheral implementation to reduce memory-write interrupts when I2S Tx FIFO level is low
** Paddle movement and paddle collision logic, testing and integration
** Software development and integration
** Final testing and integration
* Jasdip Sekhon
** Wiki project schedule planning
** PCB design and planning
** File loading method algorithm for storing loaded files in memory
** Audio track and audio effects selection for gameplay audio
** Palette manipulation for gameplay
** Software development and integration
** Final testing and integration
* [[File:Cmpe-244-isaac-pic.png |200px|thumb|left|Isaac Wahhab]]
** Wiki project schedule planning
** DAC resistor ladder design and integration
** Hardware procurement (LPC4088 board, Zigbees, VGA monitor, etc)
** VGA driving through LCD controller
** Audio tracker library port
** Digital audio output using I2S
** Palette scrolling feature implementation
** Graphics rendering
** Vertical scrolling implementation for the video buffer
** Sprite testing and implementation for the ball
** SD card peripheral driver for reading assets from SD card
** Art assets acquisition
** Hardware/software development and integration
** Gameplay logic design
** Gameplay audio and level design
** Final testing and integration

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
*09/21/2021
|
*09/27/2021
|
* Read previous projects, gather information and discuss among the group members.
* Initial ordering of parts (LPC 4088 board, 64x64 LED screen)
|
* Completed
* Completed
|-
! scope="row"| 2
|
*09/28/2021
|
*10/04/2021
|
* Submit project proposals
|
* Completed
|-
! scope="row"| 3
|
*10/05/2021
|
*10/11/2021
|
* Test if board can drive VGA
* Test varying clock rates
* Hardware cursor on VGA
* Rough display driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 4
|
* 10/12/2021
|
* 10/18/2021
|
* DAC resistor ladder
* RGB channels
* Palette shifting
* Measure memory buffer writing timing
* Wiki schedule planning
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
*10/19/2021
|
*10/25/2021
|
* Calculate upper-bound for VGA refresh timings
* Create GitLab repository for project
* Joystick functionality
* SD card driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
*10/26/2021
|
*11/01/2021
|
* Audio peripheral
* Wireless communication using Zigbee module
* PCB planning
* Accelerometer driver
* Continue SD card driver
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/02/2021

|
* 11/08/2021
|
* Continue audio peripheral
* Continue wireless communication using Zigbee module
* Continue accelerometer driver
* PCB finalizing and ordering
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 11/09/2021
|
* 11/15/2021
|
* Convert PNG data to graphics format
* Game background functionality
* Sprite functionality
* User interface
* Integrate components
* Initial testing
|
* Completed
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 11/16/2021
|
* 11/22/2021
|
* Continue everything from last week
* Game physics (wall collision and AI movement)
* Measurements for wooden case
* Art assets (audio and visual)
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 10
|
*11/23/2021
|
*11/29/2021
|

* Fine-tune user experience
* Make case
|
* Completed
* Completed
|-
! scope="row"| 11
|
* 11/30/2021
|
* 12/06/2021
|
* Continue fine-tune user experience
* Integrate hardware
|
* Completed
* Completed
|-
! scope="row"| 12
|
* 12/07/2021
|
* 12/16/2021
|
* Final testing
* Final demo
|
* Completed
* Completed
|-
|}

== Bill Of Materials ==
{| class="wikitable" style="margin-left: 0px; margin-right: auto;"
|-
! scope="col"| Item#
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| [https://www.embeddedartists.com/products/lpc4088-developers-kit LPC4088-32 Developer's Kit]
| 1
| $300.00
|-
! scope="row"| 2
| Microcontroller Board
| [http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems/page/sj2-board SJ2 Board]
| 1
| $50.00
|-
! scope="row"| 3
| Bluetooth Module
| [https://hub.digi.com/support/products/digi-xbee/digi-xbee-s1 Digi XBee-S1]
| 2
| N/A
|-
! scope="row"| 4
| DAC PCB
| [https://jlcpcb.com JLCPCB (Set of 5)]
| 1
| $15.00
|-
! scope="row"| 4
| Monitor
| Asus TFT Monitor
| 1
| N/A
|-
! scope="row"| 5
| Mini Breadboard
| [https://www.amazon.com/EL-CK-002-Electronic-Breadboard-Capacitor-Potentiometer/dp/B01ERP6WL4/ref=pd_sbs_147_2/132-6936347-2117569?_encoding=UTF8&pd_rd_i=B01ERP6WL4&pd_rd_r=6a783095-ec87-42ae-9f01-a5ebc9a44607&pd_rd_w=V2aU6&pd_rd_wg=11h1i&pf_rd_p=d66372fe-68a6-48a3-90ec-41d7f64212be&pf_rd_r=368WNKXFBBQY2FJQ4119&psc=1&refRID=368WNKXFBBQY2FJQ4119 ELEGOO Mini Breadboard]
| 1
| $13.99
|-
! scope="row"| 6
| Arcade buttons
| [https://www.amazon.com/EG-STARTS-Illuminated-Buttons-Raspberry/dp/B01N11BDX9?pd_rd_w=DiSpt&pf_rd_p=5bc65336-a8b0-4888-9e70-4d4ce2f6bd59&pf_rd_r=JCH2VQYXFPHND4XZ4FAM&pd_rd_r=8d6243dd-e1bd-4dae-9b19-be2e00f3085f&pd_rd_wg=K4lt9&pd_rd_i=B01N11BDX9&psc=1&ref_=pd_bap_d_rp_1_i EG STARTS LED Arcade Buttons]
| 2
| $11.99
|-
! scope="row"| 7
| VGA cable
| N/A
| 1
| N/A
|-
! scope="row"| 7
| SD card
| N/A
| 1
| N/A
|-
|}

== Design & Implementation ==

=== Hardware Design ===
We designed our PCB using Autodesk EAGLE based on the tutorial on PCB design during the class lecture. We ordered our PCB to be manufactured from JLCPCB. The PCB consists of an R-2R resistor ladder DAC for the purpose of converting the digital signal from the microcontroller to analog to display graphics on the VGA monitor. Since the VGA input for each color channel is 0-0.7V, the resistor ladder scaled the LPC4088 outputs into proper DAC values for the VGA to take in. Five bits are passed to each of the red, green, and blue channels of the VGA in addition to other signals, such as Horizontal sync (HSYNC) and Vertical sync (VSYNC).
 
[[File:pcbSchematic.png|500px|thumb|left|PCB Schematic]]
[[File:pcbLayout.png|500px|thumb|right|PCB Layout]]
[[File:pcbPic-min.png|500px|thumb|right|Manufactured PCB]]
[[File:pcbManufactured.png|500px|thumb|left|Soldered PCB]]
 

=== Hardware Interface ===

Two Digi XBee devices were used to communicate between the SJ2-Board controller and the LPC4088 board to control the paddle on the game. The XBee devices were programmed using the XTCU programming application provided by Digi International. The SJ2-Board used one XBee as a transmitted and the LPC4088 used another as a receiver. The two modules communicated using UART frames and a single byte represented any change of movement or button presses sent by the controller.

{| class="wikitable"
|-
! scope="col"| XCTU
! scope="col"| Transmitter
! scope="col"| Receiver
|-
! scope="row"| CH
| C
| C
|-
! scope="row"| ID
| 6969
| 6969
|-
! scope="row"| CE
| Coordinator
| Endpoint
|-
! scope="row"| Baud Rate
| 9600
| 9600
|-
|}
[[File:sj2Board.png|380px|thumb|center|SJ2 Board]]
[[File:CMPE146_F16_MW_XbeeModule.jpg|200px|thumb|center|XBee Device]]
 

Two LED buttons were used to interface with the controller as inputs. They were both set up using GPIO.
[[File:VGA LED Button.jpg|200px|thumb|center|LED Button]]
 
[[File:VGA_Controller_Block_Diagram.PNG|380px|thumb|center|Controller Block Diagram]]
 

==== I2S Peripheral and Stereo Audio Decoder ====

The LPC4088 is equipped with an I2S stereo encoder. This module provides a hardware-enabled solution for sending I2S-encoded audio data packets to stereo decoders, which produce the corresponding audio waveform. For this project, since 16-bit stereo audio playback was chosen, data packets transmitted on this interface were sent in the following format:

[[File:I2S_audio_data_word.png]]

Here, each word of data encodes a single packet of audio in two 16-bit sections: one signal level for the left channel, and another for the right. These 4 bytes are sent at a rate of 44100 Hz, resulting in a total data transmission of 176400 bytes a second. Rather than needing to be software-defined, this transmission is automatically handled by hardware using a 8-word deep FIFO queue that manages streaming out the audio data at a fixed rate, saving a significant amount of CPU cycles.

This encoded audio signal was sent to a dedicated external [https://www.nxp.com/docs/en/data-sheet/UDA1380.pdf stereo decoder], which output to a 3.5 mm audio jack connected to speakers in the VGA monitor.

==== LCD Peripheral ====

The LCD peripheral was configured for 16-bit color palette (5 bits of color data for each color level -- red, green, and blue -- and an unused intensity bit), at a resolution of 800 by 600 pixels, and with an active color palette size of 256 colors. Since the logic for driving an LCD display has the same concepts (horizontal blanking, vertical blanking, etc) and timing information for any given resolution was the same, we were able to directly drive a VGA display with the same signals and timing data (after the digital signal had been converted to analog).

[[File:LCD_%2B_VGA_Video_signal.png]]

The timing register configurations needed to achieve the requested video display was derived from a dedicated website for [http://tinyvga.com/vga-timing/800x600@60Hz VGA signal info].

==== External Memory ====

The game's visual assets and tracker audio file (both after being loaded from an SD card) and the game's video buffer required more memory than what is present natively in the microcontroller. To accommodate this, an external 32MiB memory module was used. This module was interfaced with using the external memory controller peripheral present on the board.

=== Software Design ===
==== Controller ====

There is an internal accelerometer on the SJ2 board that is utilized as the movement controls for the paddle movement. The y-axis value of the magnitude is polled 17 times in 17 milliseconds and is divided by 17 to get an average reading per that time frame. The maximum tilt value that can be read by the controller is set to 700 out of the total 1000 (highest possible value). The current tilt value is divided by 31 to get a total of 32 possible readings to represent 5 bits of magnitude.

The controller sends one byte of data to the console whenever there is any change in magnitude on the accelerometer or button changes. The byte contains all the information needed to control the paddle on the screen. The first two bits on the byte represent 0 if there is no button input and 1 if there is a button input. The third bit represents whether the controller is tilted left or right with 0 being left and 1 being right. The last five bits represent the controller accelerometer's magnitude.

[[File:VGA_UART_Frame.PNG]]

==== Audio ====

Sound design was provided to the project (in the form of music and sound effects) through the use of a [https://https://en.wikipedia.org/wiki/Music_tracker tracker audio library]. Source code was adopted from the [https://github.com/schismtracker/schismtracker Schism Tracker] library and altered with the following modifications:

- All UI logic was stripped from the source code

- All file handling logic was reworked to use the SD card reading functions, as the project does not support POSIX file calls

- The [https://www.libsdl.org/ Simple DirectMedia Layer] hooks were replaced with a FreeRTOS task that periodically called the waveform generation callback function, which was passed to the I2S peripheral hooked up to a stereo decoder.

=== Implementation ===

==== Ball Logic ====

The ball is controlled by the buttons and the controller accelerometer values. When the button is pressed, the ball is initially sent with a random velocity. The velocity is determined by a substate timer and is initially set as one of 6 possible values.

{| class="wikitable"
|-
! scope="col"| Substate Count
! scope="col"| X-Velocity
! scope="col"| Y-Velocity
|-
! scope="row"| 0
| 4
| -2
|-
! scope="row"| 1
| -4
| -2
|-
! scope="row"| 2
| 2
| -4
|-
! scope="row"| 3
| -2
| -4
|-
! scope="row"| 4
| 1
| -5
|-
! scope="row"| 5
| -1
| -5
|-
|}

The ball's velocity changes whenever it detects a collision with either the side of the screen, the bricks on the top of the level, or the paddle at the bottom. If colliding with the paddle, the ball's velocity will take into account the paddle's velocity and add it to its current velocity. This is done by adding the paddle's x-velocity and adjusting the y-velocity by normalizing the velocity vectors to have a magnitude of 5. This means the ball will always be moving with a magnitude of 5 in any direction after it hits the paddle. If the ball collides with anything other than the paddle, then the velocity of the ball is changed with no increase or decrease of magnitude. If the ball collides on the side of a wall or block, the x-velocity will reflect (go from negative to positive or vice-versa) and the y-velocity will remain constant. If the ball collides on the top or bottom of a block, the y-velocity will reflect and the x-velocity will remain constant. If it appears that the ball collides with the side and top or bottom of something, then the ball will check whichever face it will collide with first and then calculate the other face. Lastly, when the ball collides with the bottom of the screen, then the player experiences a "death" and the ball will reset on the paddle. If the ball collides with the top of the screen, the screen will scroll up for the user to play on the next level.

Whenever the ball collides with a brick, the brick will disappear. To accomplish this, there is a 2-D array of unsigned integers that contains the values of the bricks. If the integer is 0, then there is no block. If the integer is any value other than 0, then the block is filled and has collision logic. If the ball approaches a region that might have a block, then there will be a check within that 2-D to see if the block is filled or empty. If the block is filled, there will be the collision logic that changes the vector of the ball and sets the integer value in the 2-D array to 0 to represent that the block has been broken.

==== Paddle Movement ====
The paddle movement is controlled by the accelerometer data being wirelessly transmitted from the SJ-Two board to the LPC4088 board. The accelerometer data will map to 63 possible indexes for the paddle location. Originally, the paddle would jump directly into the corresponding index location depending on the accelerometer input. We later iterated on this process by implementing an algorithm to smoothly transition into these indexes. This was done by calculating a paddle velocity using the "current desired paddle position" and the previous paddle position. We adjusted the parameters of the algorithm iteratively to produce the smooth paddle movement.

==== Paddle Collision Logic ====
The position of the ball is calculated by adding delta_x and delta_y to the ball's current x and y position. For each frame, if the ball's next Y position crosses the paddle's Y position, we would check for a paddle collision. To do this, we used the delta_x and delta_y to get the ball's movement as a slope. We then substituted the slope, the ball's current position, and 1 pixel above the paddle's Y position into a point-slope equation (y2 - y1) = m(x2 - x1) to get the paddle's X position at the time of the ball reaching the pixel above paddle's Y position. After finding the location where the ball is expected to collide with the paddle, we then calculated the ball's collision magnitude and the ball's next location's magnitude. Using a ratio of these two magnitudes, we have a frame time fraction of when the ball possibly collided with the paddle. Then, by using the paddle's current position and paddle velocity, we are able to find where the paddle previously was, and measure how far the paddle had moved during that time frame. By multiplying the time frame from the ball's collision magnitude to the paddle's change in X position, we are able to find whether the paddle was under the ball at the time of the ball reaching the pixel above the paddle's Y position.

== Testing & Technical Challenges ==

=== VGA Interfacing ===
A DAC resistor ladder was designed and a corresponding custom PCB was acquired to interface the 3.3V digital LCD controller signal with the analog VGA input signal. The VGA cable takes between 0V (completely dark) and 0.7V (maximum brightness) for color pin inputs. The DAC resistor ladder was designed and implemented on a breadboard to verify the design. Validating the design on a breadboard gave us a clear vision during PCB design and planning.

=== VGA Write Timings ===
There were initial tests that were required to be done to see if using a VGA monitor as an output was a viable option for this project. We had to test memory write timings to see if they were fast enough to match the refresh rate of the monitor. Those tests would decide how much of the screen buffer we could write to during the blanking periods during the VGA refresh. This was a good learning experience for testing hardware capabilities and should be done whenever introducing any new aspects to a project.

=== Ball Collision Logic ===
The initial version of the ball collision used pixel values to see if the ball was entering the brick. The problem was that this logic did not account for the case where the ball enters the brick diagonally and the logic for that version of the collision had to be scrapped. The later iteration of the logic was changed to checking whether the ball was potentially crossing into an area that had a brick. This was done by separating the checks into a grid that represented the locations of the blocks. In this case, there was an initial check to see if the ball was crossing a boundary where there might be a new brick. If the ball is crossing the x-boundary and not the y-boundary, then there would be a collision check to see if it collided with a block. If it is crossing the y-boundary and not the x-boundary, once again then there would be a collision check to see if it collided with a block. If it is crossing both the x-boundary and y-boundary, then there would be a check to see which boundary the ball would cross first. This new logic allowed for us to account for the case where the block enters the block diagonally. This taught us that sometimes it is beneficial to scrap a previous iteration and use a new approach to solving a problem because it may lead to better results.

=== Paddle Movement ===
One issue we discovered was the paddle would slow down as it approached the walls. This was due to the difference in "current desired paddle position" and the previous paddle position becoming smaller as the walls were approached, leading to a smaller paddle velocity. To account for this, whenever the accelerometer sent data that mapped to the minimum or maximum paddle index, we would increase the distance of the "current desired paddle position" so the paddle velocity would not slow the paddle down as it approached the walls.

=== Paddle Collision Logic ===
During final integration and testing, one of the biggest issues we encountered was a bug where the ball would not detect a collision with the paddle. The ball would phase right through the paddle instead of bouncing off of it. One of the great ideas that we thought of was to change the color of the ball during each check to see if the checks would go through. This debug check allowed us to verify that section of code that checked for paddle collisions was being skipped even though the ball's next Y position had crossed the paddle's Y position. By having this debugging method, we were able to root cause the issue.

== Conclusion ==
Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

=== Project Video ===
[https://www.youtube.com/watch?v=ipngAGlN86c SJSU CMPE 244 FALL 2021: Embedded Software Project Gameplay]

=== Project Source Code ===
* [https://gitlab.com/fayek.wahhab/sjtwo-c/-/tree/feature/i2s_dma2 Treasure Dive Baseline for Demo Source Code]
* [https://gitlab.com/hobriandh/sjtwo-c/-/tree/final_controller Controller for Demo Source Code]

== References ==
=== Acknowledgement ===
We would like to acknowledge and thank our instructor Preetpal Kang for sharing his knowledge in embedded software development using FreeRTOS.

=== Appendix ===
*[https://gitlab.com/sjtwo-c-dev/sjtwo-c/-/blob/master/LPC408X_7X%20User%20Manual.pdf LPC408x/407x Datasheet]
*[https://www.freertos.org/Documentation/RTOS_book.html FreeRTOS Documentation]
*[http://elm-chan.org/fsw/ff/00index_e.html FatFs - Generic FAT Filesystem Module]
*[https://www.digi.com/resources/library/data-sheets/ds_xbeemultipointmodules Xbee Datasheet]
*[http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems Embedded Drivers & Real Time Operating Systems Lessons]

File:Cmpe-244-isaac-pic.png

2021-12-18T12:01:01Z

Proj user1:

F21: Treasure Dive

2021-12-18T11:59:05Z

Proj user1: /* Team Members & Responsibilities */

== '''Treasure Dive''' ==
[[File:mermaid.png|thumb|500px|right|Game Screen Capture]]
[[File:Cmpe-244-controller-pic.png|thumb|600px|right|Game Controller]]

== Abstract ==
Treasure Dive is a modern one player game inspired by the popular arcade game Breakout, which was released by Atari in 1976. The ancients rambled about an unconfirmed tale of a shipwreck that saw no survivors. As an adventurous and skilled free diver, you are seeking treasure buried deep in an ocean abyss. You dove down all the way to the seafloor and found the buried treasure but need to make it back to the surface to see another day! The player uses a wireless controller to control an on-screen paddle to break bricks on the top of the screen. When enough bricks are broken, a passage is revealed, allowing you to get closer to the surface.

== Objectives & Introduction ==
The objective of this project is to interface an LPC4088 to a VGA monitor by leveraging the LCD controller and use an accelerometer as the sensor for the player to control the paddle.
Two Zigbees RF modules will be integrated for wireless communication between the controller and LPC4088 board via UART protocol.
Digital audio output will be enabled by developing an I2S driver and porting an audio tracker library.
The General Purpose Direct Memory Access (GPDMA) controller memory-to-memory function will be utilized to decrease memory write times when updating the video buffer.
The DMA controller's memory-to-peripheral function will help reduce the frequent I2S interrupts when loading digital audio contents.
An SD card peripheral driver to facilitate memory exchange between an SD card will also be implemented.

=== Team Members & Responsibilities ===
* [[File:Cmpe-244-brian-pic.png ‎]]
* Brian Ho
** Wiki project schedule planning
** Interfacing the accelerometer
** Digital to Analog Conversion resistor ladder design used to interface digital LCD controller signal with analog VGA port signal
** PCB planning and acquisition
** Procurement of controller hardware
** Wireless communication via Zigbees RF modules and UART protocol
** Wireless controller logic and integration
** Ball movement and wall/brick collision logic
** Hardware/software development and integration
** Final testing and integration
* [https://www.linkedin.com/in/william-lai-525681139 Billy Lai]
** Wiki project schedule planning
** Increased CPU clock rate from 96 MHz to 120 MHz
** Initial memory write timing and measurements to estimate memory write capacity
** DMA driver memory-to-memory implementation for transferring graphics data to video buffer
** I2S with DMA memory-to-peripheral implementation to reduce memory-write interrupts when I2S Tx FIFO level is low
** Paddle movement and paddle collision logic, testing and integration
** Software development and integration
** Final testing and integration
* Jasdip Sekhon
** Wiki project schedule planning
** PCB design and planning
** File loading method algorithm for storing loaded files in memory
** Audio track and audio effects selection for gameplay audio
** Palette manipulation for gameplay
** Software development and integration
** Final testing and integration
* Isaac Wahhab
** Wiki project schedule planning
** DAC resistor ladder design and integration
** Hardware procurement (LPC4088 board, Zigbees, VGA monitor, etc)
** VGA driving through LCD controller
** Audio tracker library port
** Digital audio output using I2S
** Palette scrolling feature implementation
** Graphics rendering
** Vertical scrolling implementation for the video buffer
** Sprite testing and implementation for the ball
** SD card peripheral driver for reading assets from SD card
** Art assets acquisition
** Hardware/software development and integration
** Gameplay logic design
** Gameplay audio and level design
** Final testing and integration

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
*09/21/2021
|
*09/27/2021
|
* Read previous projects, gather information and discuss among the group members.
* Initial ordering of parts (LPC 4088 board, 64x64 LED screen)
|
* Completed
* Completed
|-
! scope="row"| 2
|
*09/28/2021
|
*10/04/2021
|
* Submit project proposals
|
* Completed
|-
! scope="row"| 3
|
*10/05/2021
|
*10/11/2021
|
* Test if board can drive VGA
* Test varying clock rates
* Hardware cursor on VGA
* Rough display driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 4
|
* 10/12/2021
|
* 10/18/2021
|
* DAC resistor ladder
* RGB channels
* Palette shifting
* Measure memory buffer writing timing
* Wiki schedule planning
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
*10/19/2021
|
*10/25/2021
|
* Calculate upper-bound for VGA refresh timings
* Create GitLab repository for project
* Joystick functionality
* SD card driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
*10/26/2021
|
*11/01/2021
|
* Audio peripheral
* Wireless communication using Zigbee module
* PCB planning
* Accelerometer driver
* Continue SD card driver
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/02/2021

|
* 11/08/2021
|
* Continue audio peripheral
* Continue wireless communication using Zigbee module
* Continue accelerometer driver
* PCB finalizing and ordering
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 11/09/2021
|
* 11/15/2021
|
* Convert PNG data to graphics format
* Game background functionality
* Sprite functionality
* User interface
* Integrate components
* Initial testing
|
* Completed
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 11/16/2021
|
* 11/22/2021
|
* Continue everything from last week
* Game physics (wall collision and AI movement)
* Measurements for wooden case
* Art assets (audio and visual)
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 10
|
*11/23/2021
|
*11/29/2021
|

* Fine-tune user experience
* Make case
|
* Completed
* Completed
|-
! scope="row"| 11
|
* 11/30/2021
|
* 12/06/2021
|
* Continue fine-tune user experience
* Integrate hardware
|
* Completed
* Completed
|-
! scope="row"| 12
|
* 12/07/2021
|
* 12/16/2021
|
* Final testing
* Final demo
|
* Completed
* Completed
|-
|}

== Bill Of Materials ==
{| class="wikitable" style="margin-left: 0px; margin-right: auto;"
|-
! scope="col"| Item#
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| [https://www.embeddedartists.com/products/lpc4088-developers-kit LPC4088-32 Developer's Kit]
| 1
| $300.00
|-
! scope="row"| 2
| Microcontroller Board
| [http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems/page/sj2-board SJ2 Board]
| 1
| $50.00
|-
! scope="row"| 3
| Bluetooth Module
| [https://hub.digi.com/support/products/digi-xbee/digi-xbee-s1 Digi XBee-S1]
| 2
| N/A
|-
! scope="row"| 4
| DAC PCB
| [https://jlcpcb.com JLCPCB (Set of 5)]
| 1
| $15.00
|-
! scope="row"| 4
| Monitor
| Asus TFT Monitor
| 1
| N/A
|-
! scope="row"| 5
| Mini Breadboard
| [https://www.amazon.com/EL-CK-002-Electronic-Breadboard-Capacitor-Potentiometer/dp/B01ERP6WL4/ref=pd_sbs_147_2/132-6936347-2117569?_encoding=UTF8&pd_rd_i=B01ERP6WL4&pd_rd_r=6a783095-ec87-42ae-9f01-a5ebc9a44607&pd_rd_w=V2aU6&pd_rd_wg=11h1i&pf_rd_p=d66372fe-68a6-48a3-90ec-41d7f64212be&pf_rd_r=368WNKXFBBQY2FJQ4119&psc=1&refRID=368WNKXFBBQY2FJQ4119 ELEGOO Mini Breadboard]
| 1
| $13.99
|-
! scope="row"| 6
| Arcade buttons
| [https://www.amazon.com/EG-STARTS-Illuminated-Buttons-Raspberry/dp/B01N11BDX9?pd_rd_w=DiSpt&pf_rd_p=5bc65336-a8b0-4888-9e70-4d4ce2f6bd59&pf_rd_r=JCH2VQYXFPHND4XZ4FAM&pd_rd_r=8d6243dd-e1bd-4dae-9b19-be2e00f3085f&pd_rd_wg=K4lt9&pd_rd_i=B01N11BDX9&psc=1&ref_=pd_bap_d_rp_1_i EG STARTS LED Arcade Buttons]
| 2
| $11.99
|-
! scope="row"| 7
| VGA cable
| N/A
| 1
| N/A
|-
! scope="row"| 7
| SD card
| N/A
| 1
| N/A
|-
|}

== Design & Implementation ==

=== Hardware Design ===
We designed our PCB using Autodesk EAGLE based on the tutorial on PCB design during the class lecture. We ordered our PCB to be manufactured from JLCPCB. The PCB consists of an R-2R resistor ladder DAC for the purpose of converting the digital signal from the microcontroller to analog to display graphics on the VGA monitor. Since the VGA input for each color channel is 0-0.7V, the resistor ladder scaled the LPC4088 outputs into proper DAC values for the VGA to take in. Five bits are passed to each of the red, green, and blue channels of the VGA in addition to other signals, such as Horizontal sync (HSYNC) and Vertical sync (VSYNC).
 
[[File:pcbSchematic.png|500px|thumb|left|PCB Schematic]]
[[File:pcbLayout.png|500px|thumb|right|PCB Layout]]
[[File:pcbPic-min.png|500px|thumb|right|Manufactured PCB]]
[[File:pcbManufactured.png|500px|thumb|left|Soldered PCB]]
 

=== Hardware Interface ===

Two Digi XBee devices were used to communicate between the SJ2-Board controller and the LPC4088 board to control the paddle on the game. The XBee devices were programmed using the XTCU programming application provided by Digi International. The SJ2-Board used one XBee as a transmitted and the LPC4088 used another as a receiver. The two modules communicated using UART frames and a single byte represented any change of movement or button presses sent by the controller.

{| class="wikitable"
|-
! scope="col"| XCTU
! scope="col"| Transmitter
! scope="col"| Receiver
|-
! scope="row"| CH
| C
| C
|-
! scope="row"| ID
| 6969
| 6969
|-
! scope="row"| CE
| Coordinator
| Endpoint
|-
! scope="row"| Baud Rate
| 9600
| 9600
|-
|}
[[File:sj2Board.png|380px|thumb|center|SJ2 Board]]
[[File:CMPE146_F16_MW_XbeeModule.jpg|200px|thumb|center|XBee Device]]
 

Two LED buttons were used to interface with the controller as inputs. They were both set up using GPIO.
[[File:VGA LED Button.jpg|200px|thumb|center|LED Button]]
 
[[File:VGA_Controller_Block_Diagram.PNG|380px|thumb|center|Controller Block Diagram]]
 

==== I2S Peripheral and Stereo Audio Decoder ====

The LPC4088 is equipped with an I2S stereo encoder. This module provides a hardware-enabled solution for sending I2S-encoded audio data packets to stereo decoders, which produce the corresponding audio waveform. For this project, since 16-bit stereo audio playback was chosen, data packets transmitted on this interface were sent in the following format:

[[File:I2S_audio_data_word.png]]

Here, each word of data encodes a single packet of audio in two 16-bit sections: one signal level for the left channel, and another for the right. These 4 bytes are sent at a rate of 44100 Hz, resulting in a total data transmission of 176400 bytes a second. Rather than needing to be software-defined, this transmission is automatically handled by hardware using a 8-word deep FIFO queue that manages streaming out the audio data at a fixed rate, saving a significant amount of CPU cycles.

This encoded audio signal was sent to a dedicated external [https://www.nxp.com/docs/en/data-sheet/UDA1380.pdf stereo decoder], which output to a 3.5 mm audio jack connected to speakers in the VGA monitor.

==== LCD Peripheral ====

The LCD peripheral was configured for 16-bit color palette (5 bits of color data for each color level -- red, green, and blue -- and an unused intensity bit), at a resolution of 800 by 600 pixels, and with an active color palette size of 256 colors. Since the logic for driving an LCD display has the same concepts (horizontal blanking, vertical blanking, etc) and timing information for any given resolution was the same, we were able to directly drive a VGA display with the same signals and timing data (after the digital signal had been converted to analog).

[[File:LCD_%2B_VGA_Video_signal.png]]

The timing register configurations needed to achieve the requested video display was derived from a dedicated website for [http://tinyvga.com/vga-timing/800x600@60Hz VGA signal info].

==== External Memory ====

The game's visual assets and tracker audio file (both after being loaded from an SD card) and the game's video buffer required more memory than what is present natively in the microcontroller. To accommodate this, an external 32MiB memory module was used. This module was interfaced with using the external memory controller peripheral present on the board.

=== Software Design ===
==== Controller ====

There is an internal accelerometer on the SJ2 board that is utilized as the movement controls for the paddle movement. The y-axis value of the magnitude is polled 17 times in 17 milliseconds and is divided by 17 to get an average reading per that time frame. The maximum tilt value that can be read by the controller is set to 700 out of the total 1000 (highest possible value). The current tilt value is divided by 31 to get a total of 32 possible readings to represent 5 bits of magnitude.

The controller sends one byte of data to the console whenever there is any change in magnitude on the accelerometer or button changes. The byte contains all the information needed to control the paddle on the screen. The first two bits on the byte represent 0 if there is no button input and 1 if there is a button input. The third bit represents whether the controller is tilted left or right with 0 being left and 1 being right. The last five bits represent the controller accelerometer's magnitude.

[[File:VGA_UART_Frame.PNG]]

==== Audio ====

Sound design was provided to the project (in the form of music and sound effects) through the use of a [https://https://en.wikipedia.org/wiki/Music_tracker tracker audio library]. Source code was adopted from the [https://github.com/schismtracker/schismtracker Schism Tracker] library and altered with the following modifications:

- All UI logic was stripped from the source code

- All file handling logic was reworked to use the SD card reading functions, as the project does not support POSIX file calls

- The [https://www.libsdl.org/ Simple DirectMedia Layer] hooks were replaced with a FreeRTOS task that periodically called the waveform generation callback function, which was passed to the I2S peripheral hooked up to a stereo decoder.

=== Implementation ===

==== Ball Logic ====

The ball is controlled by the buttons and the controller accelerometer values. When the button is pressed, the ball is initially sent with a random velocity. The velocity is determined by a substate timer and is initially set as one of 6 possible values.

{| class="wikitable"
|-
! scope="col"| Substate Count
! scope="col"| X-Velocity
! scope="col"| Y-Velocity
|-
! scope="row"| 0
| 4
| -2
|-
! scope="row"| 1
| -4
| -2
|-
! scope="row"| 2
| 2
| -4
|-
! scope="row"| 3
| -2
| -4
|-
! scope="row"| 4
| 1
| -5
|-
! scope="row"| 5
| -1
| -5
|-
|}

The ball's velocity changes whenever it detects a collision with either the side of the screen, the bricks on the top of the level, or the paddle at the bottom. If colliding with the paddle, the ball's velocity will take into account the paddle's velocity and add it to its current velocity. This is done by adding the paddle's x-velocity and adjusting the y-velocity by normalizing the velocity vectors to have a magnitude of 5. This means the ball will always be moving with a magnitude of 5 in any direction after it hits the paddle. If the ball collides with anything other than the paddle, then the velocity of the ball is changed with no increase or decrease of magnitude. If the ball collides on the side of a wall or block, the x-velocity will reflect (go from negative to positive or vice-versa) and the y-velocity will remain constant. If the ball collides on the top or bottom of a block, the y-velocity will reflect and the x-velocity will remain constant. If it appears that the ball collides with the side and top or bottom of something, then the ball will check whichever face it will collide with first and then calculate the other face. Lastly, when the ball collides with the bottom of the screen, then the player experiences a "death" and the ball will reset on the paddle. If the ball collides with the top of the screen, the screen will scroll up for the user to play on the next level.

Whenever the ball collides with a brick, the brick will disappear. To accomplish this, there is a 2-D array of unsigned integers that contains the values of the bricks. If the integer is 0, then there is no block. If the integer is any value other than 0, then the block is filled and has collision logic. If the ball approaches a region that might have a block, then there will be a check within that 2-D to see if the block is filled or empty. If the block is filled, there will be the collision logic that changes the vector of the ball and sets the integer value in the 2-D array to 0 to represent that the block has been broken.

==== Paddle Movement ====
The paddle movement is controlled by the accelerometer data being wirelessly transmitted from the SJ-Two board to the LPC4088 board. The accelerometer data will map to 63 possible indexes for the paddle location. Originally, the paddle would jump directly into the corresponding index location depending on the accelerometer input. We later iterated on this process by implementing an algorithm to smoothly transition into these indexes. This was done by calculating a paddle velocity using the "current desired paddle position" and the previous paddle position. We adjusted the parameters of the algorithm iteratively to produce the smooth paddle movement.

==== Paddle Collision Logic ====
The position of the ball is calculated by adding delta_x and delta_y to the ball's current x and y position. For each frame, if the ball's next Y position crosses the paddle's Y position, we would check for a paddle collision. To do this, we used the delta_x and delta_y to get the ball's movement as a slope. We then substituted the slope, the ball's current position, and 1 pixel above the paddle's Y position into a point-slope equation (y2 - y1) = m(x2 - x1) to get the paddle's X position at the time of the ball reaching the pixel above paddle's Y position. After finding the location where the ball is expected to collide with the paddle, we then calculated the ball's collision magnitude and the ball's next location's magnitude. Using a ratio of these two magnitudes, we have a frame time fraction of when the ball possibly collided with the paddle. Then, by using the paddle's current position and paddle velocity, we are able to find where the paddle previously was, and measure how far the paddle had moved during that time frame. By multiplying the time frame from the ball's collision magnitude to the paddle's change in X position, we are able to find whether the paddle was under the ball at the time of the ball reaching the pixel above the paddle's Y position.

== Testing & Technical Challenges ==

=== VGA Interfacing ===
A DAC resistor ladder was designed and a corresponding custom PCB was acquired to interface the 3.3V digital LCD controller signal with the analog VGA input signal. The VGA cable takes between 0V (completely dark) and 0.7V (maximum brightness) for color pin inputs. The DAC resistor ladder was designed and implemented on a breadboard to verify the design. Validating the design on a breadboard gave us a clear vision during PCB design and planning.

=== VGA Write Timings ===
There were initial tests that were required to be done to see if using a VGA monitor as an output was a viable option for this project. We had to test memory write timings to see if they were fast enough to match the refresh rate of the monitor. Those tests would decide how much of the screen buffer we could write to during the blanking periods during the VGA refresh. This was a good learning experience for testing hardware capabilities and should be done whenever introducing any new aspects to a project.

=== Ball Collision Logic ===
The initial version of the ball collision used pixel values to see if the ball was entering the brick. The problem was that this logic did not account for the case where the ball enters the brick diagonally and the logic for that version of the collision had to be scrapped. The later iteration of the logic was changed to checking whether the ball was potentially crossing into an area that had a brick. This was done by separating the checks into a grid that represented the locations of the blocks. In this case, there was an initial check to see if the ball was crossing a boundary where there might be a new brick. If the ball is crossing the x-boundary and not the y-boundary, then there would be a collision check to see if it collided with a block. If it is crossing the y-boundary and not the x-boundary, once again then there would be a collision check to see if it collided with a block. If it is crossing both the x-boundary and y-boundary, then there would be a check to see which boundary the ball would cross first. This new logic allowed for us to account for the case where the block enters the block diagonally. This taught us that sometimes it is beneficial to scrap a previous iteration and use a new approach to solving a problem because it may lead to better results.

=== Paddle Movement ===
One issue we discovered was the paddle would slow down as it approached the walls. This was due to the difference in "current desired paddle position" and the previous paddle position becoming smaller as the walls were approached, leading to a smaller paddle velocity. To account for this, whenever the accelerometer sent data that mapped to the minimum or maximum paddle index, we would increase the distance of the "current desired paddle position" so the paddle velocity would not slow the paddle down as it approached the walls.

=== Paddle Collision Logic ===
During final integration and testing, one of the biggest issues we encountered was a bug where the ball would not detect a collision with the paddle. The ball would phase right through the paddle instead of bouncing off of it. One of the great ideas that we thought of was to change the color of the ball during each check to see if the checks would go through. This debug check allowed us to verify that section of code that checked for paddle collisions was being skipped even though the ball's next Y position had crossed the paddle's Y position. By having this debugging method, we were able to root cause the issue.

== Conclusion ==
Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

=== Project Video ===
[https://www.youtube.com/watch?v=ipngAGlN86c SJSU CMPE 244 FALL 2021: Embedded Software Project Gameplay]

=== Project Source Code ===
* [https://gitlab.com/fayek.wahhab/sjtwo-c/-/tree/feature/i2s_dma2 Treasure Dive Baseline for Demo Source Code]
* [https://gitlab.com/hobriandh/sjtwo-c/-/tree/final_controller Controller for Demo Source Code]

== References ==
=== Acknowledgement ===
We would like to acknowledge and thank our instructor Preetpal Kang for sharing his knowledge in embedded software development using FreeRTOS.

=== Appendix ===
*[https://gitlab.com/sjtwo-c-dev/sjtwo-c/-/blob/master/LPC408X_7X%20User%20Manual.pdf LPC408x/407x Datasheet]
*[https://www.freertos.org/Documentation/RTOS_book.html FreeRTOS Documentation]
*[http://elm-chan.org/fsw/ff/00index_e.html FatFs - Generic FAT Filesystem Module]
*[https://www.digi.com/resources/library/data-sheets/ds_xbeemultipointmodules Xbee Datasheet]
*[http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems Embedded Drivers & Real Time Operating Systems Lessons]

F21: Treasure Dive

2021-12-18T11:58:36Z

Proj user1: /* Team Members & Responsibilities */

== '''Treasure Dive''' ==
[[File:mermaid.png|thumb|500px|right|Game Screen Capture]]
[[File:Cmpe-244-controller-pic.png|thumb|600px|right|Game Controller]]

== Abstract ==
Treasure Dive is a modern one player game inspired by the popular arcade game Breakout, which was released by Atari in 1976. The ancients rambled about an unconfirmed tale of a shipwreck that saw no survivors. As an adventurous and skilled free diver, you are seeking treasure buried deep in an ocean abyss. You dove down all the way to the seafloor and found the buried treasure but need to make it back to the surface to see another day! The player uses a wireless controller to control an on-screen paddle to break bricks on the top of the screen. When enough bricks are broken, a passage is revealed, allowing you to get closer to the surface.

== Objectives & Introduction ==
The objective of this project is to interface an LPC4088 to a VGA monitor by leveraging the LCD controller and use an accelerometer as the sensor for the player to control the paddle.
Two Zigbees RF modules will be integrated for wireless communication between the controller and LPC4088 board via UART protocol.
Digital audio output will be enabled by developing an I2S driver and porting an audio tracker library.
The General Purpose Direct Memory Access (GPDMA) controller memory-to-memory function will be utilized to decrease memory write times when updating the video buffer.
The DMA controller's memory-to-peripheral function will help reduce the frequent I2S interrupts when loading digital audio contents.
An SD card peripheral driver to facilitate memory exchange between an SD card will also be implemented.

=== Team Members & Responsibilities ===
* [[File:Cmpe-244-brian-pic.png ‎Brian Ho]]
** Wiki project schedule planning
** Interfacing the accelerometer
** Digital to Analog Conversion resistor ladder design used to interface digital LCD controller signal with analog VGA port signal
** PCB planning and acquisition
** Procurement of controller hardware
** Wireless communication via Zigbees RF modules and UART protocol
** Wireless controller logic and integration
** Ball movement and wall/brick collision logic
** Hardware/software development and integration
** Final testing and integration
* [https://www.linkedin.com/in/william-lai-525681139 Billy Lai]
** Wiki project schedule planning
** Increased CPU clock rate from 96 MHz to 120 MHz
** Initial memory write timing and measurements to estimate memory write capacity
** DMA driver memory-to-memory implementation for transferring graphics data to video buffer
** I2S with DMA memory-to-peripheral implementation to reduce memory-write interrupts when I2S Tx FIFO level is low
** Paddle movement and paddle collision logic, testing and integration
** Software development and integration
** Final testing and integration
* Jasdip Sekhon
** Wiki project schedule planning
** PCB design and planning
** File loading method algorithm for storing loaded files in memory
** Audio track and audio effects selection for gameplay audio
** Palette manipulation for gameplay
** Software development and integration
** Final testing and integration
* Isaac Wahhab
** Wiki project schedule planning
** DAC resistor ladder design and integration
** Hardware procurement (LPC4088 board, Zigbees, VGA monitor, etc)
** VGA driving through LCD controller
** Audio tracker library port
** Digital audio output using I2S
** Palette scrolling feature implementation
** Graphics rendering
** Vertical scrolling implementation for the video buffer
** Sprite testing and implementation for the ball
** SD card peripheral driver for reading assets from SD card
** Art assets acquisition
** Hardware/software development and integration
** Gameplay logic design
** Gameplay audio and level design
** Final testing and integration

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
*09/21/2021
|
*09/27/2021
|
* Read previous projects, gather information and discuss among the group members.
* Initial ordering of parts (LPC 4088 board, 64x64 LED screen)
|
* Completed
* Completed
|-
! scope="row"| 2
|
*09/28/2021
|
*10/04/2021
|
* Submit project proposals
|
* Completed
|-
! scope="row"| 3
|
*10/05/2021
|
*10/11/2021
|
* Test if board can drive VGA
* Test varying clock rates
* Hardware cursor on VGA
* Rough display driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 4
|
* 10/12/2021
|
* 10/18/2021
|
* DAC resistor ladder
* RGB channels
* Palette shifting
* Measure memory buffer writing timing
* Wiki schedule planning
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
*10/19/2021
|
*10/25/2021
|
* Calculate upper-bound for VGA refresh timings
* Create GitLab repository for project
* Joystick functionality
* SD card driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
*10/26/2021
|
*11/01/2021
|
* Audio peripheral
* Wireless communication using Zigbee module
* PCB planning
* Accelerometer driver
* Continue SD card driver
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/02/2021

|
* 11/08/2021
|
* Continue audio peripheral
* Continue wireless communication using Zigbee module
* Continue accelerometer driver
* PCB finalizing and ordering
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 11/09/2021
|
* 11/15/2021
|
* Convert PNG data to graphics format
* Game background functionality
* Sprite functionality
* User interface
* Integrate components
* Initial testing
|
* Completed
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 11/16/2021
|
* 11/22/2021
|
* Continue everything from last week
* Game physics (wall collision and AI movement)
* Measurements for wooden case
* Art assets (audio and visual)
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 10
|
*11/23/2021
|
*11/29/2021
|

* Fine-tune user experience
* Make case
|
* Completed
* Completed
|-
! scope="row"| 11
|
* 11/30/2021
|
* 12/06/2021
|
* Continue fine-tune user experience
* Integrate hardware
|
* Completed
* Completed
|-
! scope="row"| 12
|
* 12/07/2021
|
* 12/16/2021
|
* Final testing
* Final demo
|
* Completed
* Completed
|-
|}

== Bill Of Materials ==
{| class="wikitable" style="margin-left: 0px; margin-right: auto;"
|-
! scope="col"| Item#
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| [https://www.embeddedartists.com/products/lpc4088-developers-kit LPC4088-32 Developer's Kit]
| 1
| $300.00
|-
! scope="row"| 2
| Microcontroller Board
| [http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems/page/sj2-board SJ2 Board]
| 1
| $50.00
|-
! scope="row"| 3
| Bluetooth Module
| [https://hub.digi.com/support/products/digi-xbee/digi-xbee-s1 Digi XBee-S1]
| 2
| N/A
|-
! scope="row"| 4
| DAC PCB
| [https://jlcpcb.com JLCPCB (Set of 5)]
| 1
| $15.00
|-
! scope="row"| 4
| Monitor
| Asus TFT Monitor
| 1
| N/A
|-
! scope="row"| 5
| Mini Breadboard
| [https://www.amazon.com/EL-CK-002-Electronic-Breadboard-Capacitor-Potentiometer/dp/B01ERP6WL4/ref=pd_sbs_147_2/132-6936347-2117569?_encoding=UTF8&pd_rd_i=B01ERP6WL4&pd_rd_r=6a783095-ec87-42ae-9f01-a5ebc9a44607&pd_rd_w=V2aU6&pd_rd_wg=11h1i&pf_rd_p=d66372fe-68a6-48a3-90ec-41d7f64212be&pf_rd_r=368WNKXFBBQY2FJQ4119&psc=1&refRID=368WNKXFBBQY2FJQ4119 ELEGOO Mini Breadboard]
| 1
| $13.99
|-
! scope="row"| 6
| Arcade buttons
| [https://www.amazon.com/EG-STARTS-Illuminated-Buttons-Raspberry/dp/B01N11BDX9?pd_rd_w=DiSpt&pf_rd_p=5bc65336-a8b0-4888-9e70-4d4ce2f6bd59&pf_rd_r=JCH2VQYXFPHND4XZ4FAM&pd_rd_r=8d6243dd-e1bd-4dae-9b19-be2e00f3085f&pd_rd_wg=K4lt9&pd_rd_i=B01N11BDX9&psc=1&ref_=pd_bap_d_rp_1_i EG STARTS LED Arcade Buttons]
| 2
| $11.99
|-
! scope="row"| 7
| VGA cable
| N/A
| 1
| N/A
|-
! scope="row"| 7
| SD card
| N/A
| 1
| N/A
|-
|}

== Design & Implementation ==

=== Hardware Design ===
We designed our PCB using Autodesk EAGLE based on the tutorial on PCB design during the class lecture. We ordered our PCB to be manufactured from JLCPCB. The PCB consists of an R-2R resistor ladder DAC for the purpose of converting the digital signal from the microcontroller to analog to display graphics on the VGA monitor. Since the VGA input for each color channel is 0-0.7V, the resistor ladder scaled the LPC4088 outputs into proper DAC values for the VGA to take in. Five bits are passed to each of the red, green, and blue channels of the VGA in addition to other signals, such as Horizontal sync (HSYNC) and Vertical sync (VSYNC).
 
[[File:pcbSchematic.png|500px|thumb|left|PCB Schematic]]
[[File:pcbLayout.png|500px|thumb|right|PCB Layout]]
[[File:pcbPic-min.png|500px|thumb|right|Manufactured PCB]]
[[File:pcbManufactured.png|500px|thumb|left|Soldered PCB]]
 

=== Hardware Interface ===

Two Digi XBee devices were used to communicate between the SJ2-Board controller and the LPC4088 board to control the paddle on the game. The XBee devices were programmed using the XTCU programming application provided by Digi International. The SJ2-Board used one XBee as a transmitted and the LPC4088 used another as a receiver. The two modules communicated using UART frames and a single byte represented any change of movement or button presses sent by the controller.

{| class="wikitable"
|-
! scope="col"| XCTU
! scope="col"| Transmitter
! scope="col"| Receiver
|-
! scope="row"| CH
| C
| C
|-
! scope="row"| ID
| 6969
| 6969
|-
! scope="row"| CE
| Coordinator
| Endpoint
|-
! scope="row"| Baud Rate
| 9600
| 9600
|-
|}
[[File:sj2Board.png|380px|thumb|center|SJ2 Board]]
[[File:CMPE146_F16_MW_XbeeModule.jpg|200px|thumb|center|XBee Device]]
 

Two LED buttons were used to interface with the controller as inputs. They were both set up using GPIO.
[[File:VGA LED Button.jpg|200px|thumb|center|LED Button]]
 
[[File:VGA_Controller_Block_Diagram.PNG|380px|thumb|center|Controller Block Diagram]]
 

==== I2S Peripheral and Stereo Audio Decoder ====

The LPC4088 is equipped with an I2S stereo encoder. This module provides a hardware-enabled solution for sending I2S-encoded audio data packets to stereo decoders, which produce the corresponding audio waveform. For this project, since 16-bit stereo audio playback was chosen, data packets transmitted on this interface were sent in the following format:

[[File:I2S_audio_data_word.png]]

Here, each word of data encodes a single packet of audio in two 16-bit sections: one signal level for the left channel, and another for the right. These 4 bytes are sent at a rate of 44100 Hz, resulting in a total data transmission of 176400 bytes a second. Rather than needing to be software-defined, this transmission is automatically handled by hardware using a 8-word deep FIFO queue that manages streaming out the audio data at a fixed rate, saving a significant amount of CPU cycles.

This encoded audio signal was sent to a dedicated external [https://www.nxp.com/docs/en/data-sheet/UDA1380.pdf stereo decoder], which output to a 3.5 mm audio jack connected to speakers in the VGA monitor.

==== LCD Peripheral ====

The LCD peripheral was configured for 16-bit color palette (5 bits of color data for each color level -- red, green, and blue -- and an unused intensity bit), at a resolution of 800 by 600 pixels, and with an active color palette size of 256 colors. Since the logic for driving an LCD display has the same concepts (horizontal blanking, vertical blanking, etc) and timing information for any given resolution was the same, we were able to directly drive a VGA display with the same signals and timing data (after the digital signal had been converted to analog).

[[File:LCD_%2B_VGA_Video_signal.png]]

The timing register configurations needed to achieve the requested video display was derived from a dedicated website for [http://tinyvga.com/vga-timing/800x600@60Hz VGA signal info].

==== External Memory ====

The game's visual assets and tracker audio file (both after being loaded from an SD card) and the game's video buffer required more memory than what is present natively in the microcontroller. To accommodate this, an external 32MiB memory module was used. This module was interfaced with using the external memory controller peripheral present on the board.

=== Software Design ===
==== Controller ====

There is an internal accelerometer on the SJ2 board that is utilized as the movement controls for the paddle movement. The y-axis value of the magnitude is polled 17 times in 17 milliseconds and is divided by 17 to get an average reading per that time frame. The maximum tilt value that can be read by the controller is set to 700 out of the total 1000 (highest possible value). The current tilt value is divided by 31 to get a total of 32 possible readings to represent 5 bits of magnitude.

The controller sends one byte of data to the console whenever there is any change in magnitude on the accelerometer or button changes. The byte contains all the information needed to control the paddle on the screen. The first two bits on the byte represent 0 if there is no button input and 1 if there is a button input. The third bit represents whether the controller is tilted left or right with 0 being left and 1 being right. The last five bits represent the controller accelerometer's magnitude.

[[File:VGA_UART_Frame.PNG]]

==== Audio ====

Sound design was provided to the project (in the form of music and sound effects) through the use of a [https://https://en.wikipedia.org/wiki/Music_tracker tracker audio library]. Source code was adopted from the [https://github.com/schismtracker/schismtracker Schism Tracker] library and altered with the following modifications:

- All UI logic was stripped from the source code

- All file handling logic was reworked to use the SD card reading functions, as the project does not support POSIX file calls

- The [https://www.libsdl.org/ Simple DirectMedia Layer] hooks were replaced with a FreeRTOS task that periodically called the waveform generation callback function, which was passed to the I2S peripheral hooked up to a stereo decoder.

=== Implementation ===

==== Ball Logic ====

The ball is controlled by the buttons and the controller accelerometer values. When the button is pressed, the ball is initially sent with a random velocity. The velocity is determined by a substate timer and is initially set as one of 6 possible values.

{| class="wikitable"
|-
! scope="col"| Substate Count
! scope="col"| X-Velocity
! scope="col"| Y-Velocity
|-
! scope="row"| 0
| 4
| -2
|-
! scope="row"| 1
| -4
| -2
|-
! scope="row"| 2
| 2
| -4
|-
! scope="row"| 3
| -2
| -4
|-
! scope="row"| 4
| 1
| -5
|-
! scope="row"| 5
| -1
| -5
|-
|}

The ball's velocity changes whenever it detects a collision with either the side of the screen, the bricks on the top of the level, or the paddle at the bottom. If colliding with the paddle, the ball's velocity will take into account the paddle's velocity and add it to its current velocity. This is done by adding the paddle's x-velocity and adjusting the y-velocity by normalizing the velocity vectors to have a magnitude of 5. This means the ball will always be moving with a magnitude of 5 in any direction after it hits the paddle. If the ball collides with anything other than the paddle, then the velocity of the ball is changed with no increase or decrease of magnitude. If the ball collides on the side of a wall or block, the x-velocity will reflect (go from negative to positive or vice-versa) and the y-velocity will remain constant. If the ball collides on the top or bottom of a block, the y-velocity will reflect and the x-velocity will remain constant. If it appears that the ball collides with the side and top or bottom of something, then the ball will check whichever face it will collide with first and then calculate the other face. Lastly, when the ball collides with the bottom of the screen, then the player experiences a "death" and the ball will reset on the paddle. If the ball collides with the top of the screen, the screen will scroll up for the user to play on the next level.

Whenever the ball collides with a brick, the brick will disappear. To accomplish this, there is a 2-D array of unsigned integers that contains the values of the bricks. If the integer is 0, then there is no block. If the integer is any value other than 0, then the block is filled and has collision logic. If the ball approaches a region that might have a block, then there will be a check within that 2-D to see if the block is filled or empty. If the block is filled, there will be the collision logic that changes the vector of the ball and sets the integer value in the 2-D array to 0 to represent that the block has been broken.

==== Paddle Movement ====
The paddle movement is controlled by the accelerometer data being wirelessly transmitted from the SJ-Two board to the LPC4088 board. The accelerometer data will map to 63 possible indexes for the paddle location. Originally, the paddle would jump directly into the corresponding index location depending on the accelerometer input. We later iterated on this process by implementing an algorithm to smoothly transition into these indexes. This was done by calculating a paddle velocity using the "current desired paddle position" and the previous paddle position. We adjusted the parameters of the algorithm iteratively to produce the smooth paddle movement.

==== Paddle Collision Logic ====
The position of the ball is calculated by adding delta_x and delta_y to the ball's current x and y position. For each frame, if the ball's next Y position crosses the paddle's Y position, we would check for a paddle collision. To do this, we used the delta_x and delta_y to get the ball's movement as a slope. We then substituted the slope, the ball's current position, and 1 pixel above the paddle's Y position into a point-slope equation (y2 - y1) = m(x2 - x1) to get the paddle's X position at the time of the ball reaching the pixel above paddle's Y position. After finding the location where the ball is expected to collide with the paddle, we then calculated the ball's collision magnitude and the ball's next location's magnitude. Using a ratio of these two magnitudes, we have a frame time fraction of when the ball possibly collided with the paddle. Then, by using the paddle's current position and paddle velocity, we are able to find where the paddle previously was, and measure how far the paddle had moved during that time frame. By multiplying the time frame from the ball's collision magnitude to the paddle's change in X position, we are able to find whether the paddle was under the ball at the time of the ball reaching the pixel above the paddle's Y position.

== Testing & Technical Challenges ==

=== VGA Interfacing ===
A DAC resistor ladder was designed and a corresponding custom PCB was acquired to interface the 3.3V digital LCD controller signal with the analog VGA input signal. The VGA cable takes between 0V (completely dark) and 0.7V (maximum brightness) for color pin inputs. The DAC resistor ladder was designed and implemented on a breadboard to verify the design. Validating the design on a breadboard gave us a clear vision during PCB design and planning.

=== VGA Write Timings ===
There were initial tests that were required to be done to see if using a VGA monitor as an output was a viable option for this project. We had to test memory write timings to see if they were fast enough to match the refresh rate of the monitor. Those tests would decide how much of the screen buffer we could write to during the blanking periods during the VGA refresh. This was a good learning experience for testing hardware capabilities and should be done whenever introducing any new aspects to a project.

=== Ball Collision Logic ===
The initial version of the ball collision used pixel values to see if the ball was entering the brick. The problem was that this logic did not account for the case where the ball enters the brick diagonally and the logic for that version of the collision had to be scrapped. The later iteration of the logic was changed to checking whether the ball was potentially crossing into an area that had a brick. This was done by separating the checks into a grid that represented the locations of the blocks. In this case, there was an initial check to see if the ball was crossing a boundary where there might be a new brick. If the ball is crossing the x-boundary and not the y-boundary, then there would be a collision check to see if it collided with a block. If it is crossing the y-boundary and not the x-boundary, once again then there would be a collision check to see if it collided with a block. If it is crossing both the x-boundary and y-boundary, then there would be a check to see which boundary the ball would cross first. This new logic allowed for us to account for the case where the block enters the block diagonally. This taught us that sometimes it is beneficial to scrap a previous iteration and use a new approach to solving a problem because it may lead to better results.

=== Paddle Movement ===
One issue we discovered was the paddle would slow down as it approached the walls. This was due to the difference in "current desired paddle position" and the previous paddle position becoming smaller as the walls were approached, leading to a smaller paddle velocity. To account for this, whenever the accelerometer sent data that mapped to the minimum or maximum paddle index, we would increase the distance of the "current desired paddle position" so the paddle velocity would not slow the paddle down as it approached the walls.

=== Paddle Collision Logic ===
During final integration and testing, one of the biggest issues we encountered was a bug where the ball would not detect a collision with the paddle. The ball would phase right through the paddle instead of bouncing off of it. One of the great ideas that we thought of was to change the color of the ball during each check to see if the checks would go through. This debug check allowed us to verify that section of code that checked for paddle collisions was being skipped even though the ball's next Y position had crossed the paddle's Y position. By having this debugging method, we were able to root cause the issue.

== Conclusion ==
Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

=== Project Video ===
[https://www.youtube.com/watch?v=ipngAGlN86c SJSU CMPE 244 FALL 2021: Embedded Software Project Gameplay]

=== Project Source Code ===
* [https://gitlab.com/fayek.wahhab/sjtwo-c/-/tree/feature/i2s_dma2 Treasure Dive Baseline for Demo Source Code]
* [https://gitlab.com/hobriandh/sjtwo-c/-/tree/final_controller Controller for Demo Source Code]

== References ==
=== Acknowledgement ===
We would like to acknowledge and thank our instructor Preetpal Kang for sharing his knowledge in embedded software development using FreeRTOS.

=== Appendix ===
*[https://gitlab.com/sjtwo-c-dev/sjtwo-c/-/blob/master/LPC408X_7X%20User%20Manual.pdf LPC408x/407x Datasheet]
*[https://www.freertos.org/Documentation/RTOS_book.html FreeRTOS Documentation]
*[http://elm-chan.org/fsw/ff/00index_e.html FatFs - Generic FAT Filesystem Module]
*[https://www.digi.com/resources/library/data-sheets/ds_xbeemultipointmodules Xbee Datasheet]
*[http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems Embedded Drivers & Real Time Operating Systems Lessons]

F21: Treasure Dive

2021-12-18T11:58:10Z

Proj user1: /* Team Members & Responsibilities */

== '''Treasure Dive''' ==
[[File:mermaid.png|thumb|500px|right|Game Screen Capture]]
[[File:Cmpe-244-controller-pic.png|thumb|600px|right|Game Controller]]

== Abstract ==
Treasure Dive is a modern one player game inspired by the popular arcade game Breakout, which was released by Atari in 1976. The ancients rambled about an unconfirmed tale of a shipwreck that saw no survivors. As an adventurous and skilled free diver, you are seeking treasure buried deep in an ocean abyss. You dove down all the way to the seafloor and found the buried treasure but need to make it back to the surface to see another day! The player uses a wireless controller to control an on-screen paddle to break bricks on the top of the screen. When enough bricks are broken, a passage is revealed, allowing you to get closer to the surface.

== Objectives & Introduction ==
The objective of this project is to interface an LPC4088 to a VGA monitor by leveraging the LCD controller and use an accelerometer as the sensor for the player to control the paddle.
Two Zigbees RF modules will be integrated for wireless communication between the controller and LPC4088 board via UART protocol.
Digital audio output will be enabled by developing an I2S driver and porting an audio tracker library.
The General Purpose Direct Memory Access (GPDMA) controller memory-to-memory function will be utilized to decrease memory write times when updating the video buffer.
The DMA controller's memory-to-peripheral function will help reduce the frequent I2S interrupts when loading digital audio contents.
An SD card peripheral driver to facilitate memory exchange between an SD card will also be implemented.

=== Team Members & Responsibilities ===
* [[File:Cmpe-244-brian-pic.png ‎]]
** Wiki project schedule planning
** Interfacing the accelerometer
** Digital to Analog Conversion resistor ladder design used to interface digital LCD controller signal with analog VGA port signal
** PCB planning and acquisition
** Procurement of controller hardware
** Wireless communication via Zigbees RF modules and UART protocol
** Wireless controller logic and integration
** Ball movement and wall/brick collision logic
** Hardware/software development and integration
** Final testing and integration
* [https://www.linkedin.com/in/william-lai-525681139 Billy Lai]
** Wiki project schedule planning
** Increased CPU clock rate from 96 MHz to 120 MHz
** Initial memory write timing and measurements to estimate memory write capacity
** DMA driver memory-to-memory implementation for transferring graphics data to video buffer
** I2S with DMA memory-to-peripheral implementation to reduce memory-write interrupts when I2S Tx FIFO level is low
** Paddle movement and paddle collision logic, testing and integration
** Software development and integration
** Final testing and integration
* Jasdip Sekhon
** Wiki project schedule planning
** PCB design and planning
** File loading method algorithm for storing loaded files in memory
** Audio track and audio effects selection for gameplay audio
** Palette manipulation for gameplay
** Software development and integration
** Final testing and integration
* Isaac Wahhab
** Wiki project schedule planning
** DAC resistor ladder design and integration
** Hardware procurement (LPC4088 board, Zigbees, VGA monitor, etc)
** VGA driving through LCD controller
** Audio tracker library port
** Digital audio output using I2S
** Palette scrolling feature implementation
** Graphics rendering
** Vertical scrolling implementation for the video buffer
** Sprite testing and implementation for the ball
** SD card peripheral driver for reading assets from SD card
** Art assets acquisition
** Hardware/software development and integration
** Gameplay logic design
** Gameplay audio and level design
** Final testing and integration

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
*09/21/2021
|
*09/27/2021
|
* Read previous projects, gather information and discuss among the group members.
* Initial ordering of parts (LPC 4088 board, 64x64 LED screen)
|
* Completed
* Completed
|-
! scope="row"| 2
|
*09/28/2021
|
*10/04/2021
|
* Submit project proposals
|
* Completed
|-
! scope="row"| 3
|
*10/05/2021
|
*10/11/2021
|
* Test if board can drive VGA
* Test varying clock rates
* Hardware cursor on VGA
* Rough display driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 4
|
* 10/12/2021
|
* 10/18/2021
|
* DAC resistor ladder
* RGB channels
* Palette shifting
* Measure memory buffer writing timing
* Wiki schedule planning
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
*10/19/2021
|
*10/25/2021
|
* Calculate upper-bound for VGA refresh timings
* Create GitLab repository for project
* Joystick functionality
* SD card driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
*10/26/2021
|
*11/01/2021
|
* Audio peripheral
* Wireless communication using Zigbee module
* PCB planning
* Accelerometer driver
* Continue SD card driver
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/02/2021

|
* 11/08/2021
|
* Continue audio peripheral
* Continue wireless communication using Zigbee module
* Continue accelerometer driver
* PCB finalizing and ordering
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 11/09/2021
|
* 11/15/2021
|
* Convert PNG data to graphics format
* Game background functionality
* Sprite functionality
* User interface
* Integrate components
* Initial testing
|
* Completed
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 11/16/2021
|
* 11/22/2021
|
* Continue everything from last week
* Game physics (wall collision and AI movement)
* Measurements for wooden case
* Art assets (audio and visual)
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 10
|
*11/23/2021
|
*11/29/2021
|

* Fine-tune user experience
* Make case
|
* Completed
* Completed
|-
! scope="row"| 11
|
* 11/30/2021
|
* 12/06/2021
|
* Continue fine-tune user experience
* Integrate hardware
|
* Completed
* Completed
|-
! scope="row"| 12
|
* 12/07/2021
|
* 12/16/2021
|
* Final testing
* Final demo
|
* Completed
* Completed
|-
|}

== Bill Of Materials ==
{| class="wikitable" style="margin-left: 0px; margin-right: auto;"
|-
! scope="col"| Item#
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| [https://www.embeddedartists.com/products/lpc4088-developers-kit LPC4088-32 Developer's Kit]
| 1
| $300.00
|-
! scope="row"| 2
| Microcontroller Board
| [http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems/page/sj2-board SJ2 Board]
| 1
| $50.00
|-
! scope="row"| 3
| Bluetooth Module
| [https://hub.digi.com/support/products/digi-xbee/digi-xbee-s1 Digi XBee-S1]
| 2
| N/A
|-
! scope="row"| 4
| DAC PCB
| [https://jlcpcb.com JLCPCB (Set of 5)]
| 1
| $15.00
|-
! scope="row"| 4
| Monitor
| Asus TFT Monitor
| 1
| N/A
|-
! scope="row"| 5
| Mini Breadboard
| [https://www.amazon.com/EL-CK-002-Electronic-Breadboard-Capacitor-Potentiometer/dp/B01ERP6WL4/ref=pd_sbs_147_2/132-6936347-2117569?_encoding=UTF8&pd_rd_i=B01ERP6WL4&pd_rd_r=6a783095-ec87-42ae-9f01-a5ebc9a44607&pd_rd_w=V2aU6&pd_rd_wg=11h1i&pf_rd_p=d66372fe-68a6-48a3-90ec-41d7f64212be&pf_rd_r=368WNKXFBBQY2FJQ4119&psc=1&refRID=368WNKXFBBQY2FJQ4119 ELEGOO Mini Breadboard]
| 1
| $13.99
|-
! scope="row"| 6
| Arcade buttons
| [https://www.amazon.com/EG-STARTS-Illuminated-Buttons-Raspberry/dp/B01N11BDX9?pd_rd_w=DiSpt&pf_rd_p=5bc65336-a8b0-4888-9e70-4d4ce2f6bd59&pf_rd_r=JCH2VQYXFPHND4XZ4FAM&pd_rd_r=8d6243dd-e1bd-4dae-9b19-be2e00f3085f&pd_rd_wg=K4lt9&pd_rd_i=B01N11BDX9&psc=1&ref_=pd_bap_d_rp_1_i EG STARTS LED Arcade Buttons]
| 2
| $11.99
|-
! scope="row"| 7
| VGA cable
| N/A
| 1
| N/A
|-
! scope="row"| 7
| SD card
| N/A
| 1
| N/A
|-
|}

== Design & Implementation ==

=== Hardware Design ===
We designed our PCB using Autodesk EAGLE based on the tutorial on PCB design during the class lecture. We ordered our PCB to be manufactured from JLCPCB. The PCB consists of an R-2R resistor ladder DAC for the purpose of converting the digital signal from the microcontroller to analog to display graphics on the VGA monitor. Since the VGA input for each color channel is 0-0.7V, the resistor ladder scaled the LPC4088 outputs into proper DAC values for the VGA to take in. Five bits are passed to each of the red, green, and blue channels of the VGA in addition to other signals, such as Horizontal sync (HSYNC) and Vertical sync (VSYNC).
 
[[File:pcbSchematic.png|500px|thumb|left|PCB Schematic]]
[[File:pcbLayout.png|500px|thumb|right|PCB Layout]]
[[File:pcbPic-min.png|500px|thumb|right|Manufactured PCB]]
[[File:pcbManufactured.png|500px|thumb|left|Soldered PCB]]
 

=== Hardware Interface ===

Two Digi XBee devices were used to communicate between the SJ2-Board controller and the LPC4088 board to control the paddle on the game. The XBee devices were programmed using the XTCU programming application provided by Digi International. The SJ2-Board used one XBee as a transmitted and the LPC4088 used another as a receiver. The two modules communicated using UART frames and a single byte represented any change of movement or button presses sent by the controller.

{| class="wikitable"
|-
! scope="col"| XCTU
! scope="col"| Transmitter
! scope="col"| Receiver
|-
! scope="row"| CH
| C
| C
|-
! scope="row"| ID
| 6969
| 6969
|-
! scope="row"| CE
| Coordinator
| Endpoint
|-
! scope="row"| Baud Rate
| 9600
| 9600
|-
|}
[[File:sj2Board.png|380px|thumb|center|SJ2 Board]]
[[File:CMPE146_F16_MW_XbeeModule.jpg|200px|thumb|center|XBee Device]]
 

Two LED buttons were used to interface with the controller as inputs. They were both set up using GPIO.
[[File:VGA LED Button.jpg|200px|thumb|center|LED Button]]
 
[[File:VGA_Controller_Block_Diagram.PNG|380px|thumb|center|Controller Block Diagram]]
 

==== I2S Peripheral and Stereo Audio Decoder ====

The LPC4088 is equipped with an I2S stereo encoder. This module provides a hardware-enabled solution for sending I2S-encoded audio data packets to stereo decoders, which produce the corresponding audio waveform. For this project, since 16-bit stereo audio playback was chosen, data packets transmitted on this interface were sent in the following format:

[[File:I2S_audio_data_word.png]]

Here, each word of data encodes a single packet of audio in two 16-bit sections: one signal level for the left channel, and another for the right. These 4 bytes are sent at a rate of 44100 Hz, resulting in a total data transmission of 176400 bytes a second. Rather than needing to be software-defined, this transmission is automatically handled by hardware using a 8-word deep FIFO queue that manages streaming out the audio data at a fixed rate, saving a significant amount of CPU cycles.

This encoded audio signal was sent to a dedicated external [https://www.nxp.com/docs/en/data-sheet/UDA1380.pdf stereo decoder], which output to a 3.5 mm audio jack connected to speakers in the VGA monitor.

==== LCD Peripheral ====

The LCD peripheral was configured for 16-bit color palette (5 bits of color data for each color level -- red, green, and blue -- and an unused intensity bit), at a resolution of 800 by 600 pixels, and with an active color palette size of 256 colors. Since the logic for driving an LCD display has the same concepts (horizontal blanking, vertical blanking, etc) and timing information for any given resolution was the same, we were able to directly drive a VGA display with the same signals and timing data (after the digital signal had been converted to analog).

[[File:LCD_%2B_VGA_Video_signal.png]]

The timing register configurations needed to achieve the requested video display was derived from a dedicated website for [http://tinyvga.com/vga-timing/800x600@60Hz VGA signal info].

==== External Memory ====

The game's visual assets and tracker audio file (both after being loaded from an SD card) and the game's video buffer required more memory than what is present natively in the microcontroller. To accommodate this, an external 32MiB memory module was used. This module was interfaced with using the external memory controller peripheral present on the board.

=== Software Design ===
==== Controller ====

There is an internal accelerometer on the SJ2 board that is utilized as the movement controls for the paddle movement. The y-axis value of the magnitude is polled 17 times in 17 milliseconds and is divided by 17 to get an average reading per that time frame. The maximum tilt value that can be read by the controller is set to 700 out of the total 1000 (highest possible value). The current tilt value is divided by 31 to get a total of 32 possible readings to represent 5 bits of magnitude.

The controller sends one byte of data to the console whenever there is any change in magnitude on the accelerometer or button changes. The byte contains all the information needed to control the paddle on the screen. The first two bits on the byte represent 0 if there is no button input and 1 if there is a button input. The third bit represents whether the controller is tilted left or right with 0 being left and 1 being right. The last five bits represent the controller accelerometer's magnitude.

[[File:VGA_UART_Frame.PNG]]

==== Audio ====

Sound design was provided to the project (in the form of music and sound effects) through the use of a [https://https://en.wikipedia.org/wiki/Music_tracker tracker audio library]. Source code was adopted from the [https://github.com/schismtracker/schismtracker Schism Tracker] library and altered with the following modifications:

- All UI logic was stripped from the source code

- All file handling logic was reworked to use the SD card reading functions, as the project does not support POSIX file calls

- The [https://www.libsdl.org/ Simple DirectMedia Layer] hooks were replaced with a FreeRTOS task that periodically called the waveform generation callback function, which was passed to the I2S peripheral hooked up to a stereo decoder.

=== Implementation ===

==== Ball Logic ====

The ball is controlled by the buttons and the controller accelerometer values. When the button is pressed, the ball is initially sent with a random velocity. The velocity is determined by a substate timer and is initially set as one of 6 possible values.

{| class="wikitable"
|-
! scope="col"| Substate Count
! scope="col"| X-Velocity
! scope="col"| Y-Velocity
|-
! scope="row"| 0
| 4
| -2
|-
! scope="row"| 1
| -4
| -2
|-
! scope="row"| 2
| 2
| -4
|-
! scope="row"| 3
| -2
| -4
|-
! scope="row"| 4
| 1
| -5
|-
! scope="row"| 5
| -1
| -5
|-
|}

The ball's velocity changes whenever it detects a collision with either the side of the screen, the bricks on the top of the level, or the paddle at the bottom. If colliding with the paddle, the ball's velocity will take into account the paddle's velocity and add it to its current velocity. This is done by adding the paddle's x-velocity and adjusting the y-velocity by normalizing the velocity vectors to have a magnitude of 5. This means the ball will always be moving with a magnitude of 5 in any direction after it hits the paddle. If the ball collides with anything other than the paddle, then the velocity of the ball is changed with no increase or decrease of magnitude. If the ball collides on the side of a wall or block, the x-velocity will reflect (go from negative to positive or vice-versa) and the y-velocity will remain constant. If the ball collides on the top or bottom of a block, the y-velocity will reflect and the x-velocity will remain constant. If it appears that the ball collides with the side and top or bottom of something, then the ball will check whichever face it will collide with first and then calculate the other face. Lastly, when the ball collides with the bottom of the screen, then the player experiences a "death" and the ball will reset on the paddle. If the ball collides with the top of the screen, the screen will scroll up for the user to play on the next level.

Whenever the ball collides with a brick, the brick will disappear. To accomplish this, there is a 2-D array of unsigned integers that contains the values of the bricks. If the integer is 0, then there is no block. If the integer is any value other than 0, then the block is filled and has collision logic. If the ball approaches a region that might have a block, then there will be a check within that 2-D to see if the block is filled or empty. If the block is filled, there will be the collision logic that changes the vector of the ball and sets the integer value in the 2-D array to 0 to represent that the block has been broken.

==== Paddle Movement ====
The paddle movement is controlled by the accelerometer data being wirelessly transmitted from the SJ-Two board to the LPC4088 board. The accelerometer data will map to 63 possible indexes for the paddle location. Originally, the paddle would jump directly into the corresponding index location depending on the accelerometer input. We later iterated on this process by implementing an algorithm to smoothly transition into these indexes. This was done by calculating a paddle velocity using the "current desired paddle position" and the previous paddle position. We adjusted the parameters of the algorithm iteratively to produce the smooth paddle movement.

==== Paddle Collision Logic ====
The position of the ball is calculated by adding delta_x and delta_y to the ball's current x and y position. For each frame, if the ball's next Y position crosses the paddle's Y position, we would check for a paddle collision. To do this, we used the delta_x and delta_y to get the ball's movement as a slope. We then substituted the slope, the ball's current position, and 1 pixel above the paddle's Y position into a point-slope equation (y2 - y1) = m(x2 - x1) to get the paddle's X position at the time of the ball reaching the pixel above paddle's Y position. After finding the location where the ball is expected to collide with the paddle, we then calculated the ball's collision magnitude and the ball's next location's magnitude. Using a ratio of these two magnitudes, we have a frame time fraction of when the ball possibly collided with the paddle. Then, by using the paddle's current position and paddle velocity, we are able to find where the paddle previously was, and measure how far the paddle had moved during that time frame. By multiplying the time frame from the ball's collision magnitude to the paddle's change in X position, we are able to find whether the paddle was under the ball at the time of the ball reaching the pixel above the paddle's Y position.

== Testing & Technical Challenges ==

=== VGA Interfacing ===
A DAC resistor ladder was designed and a corresponding custom PCB was acquired to interface the 3.3V digital LCD controller signal with the analog VGA input signal. The VGA cable takes between 0V (completely dark) and 0.7V (maximum brightness) for color pin inputs. The DAC resistor ladder was designed and implemented on a breadboard to verify the design. Validating the design on a breadboard gave us a clear vision during PCB design and planning.

=== VGA Write Timings ===
There were initial tests that were required to be done to see if using a VGA monitor as an output was a viable option for this project. We had to test memory write timings to see if they were fast enough to match the refresh rate of the monitor. Those tests would decide how much of the screen buffer we could write to during the blanking periods during the VGA refresh. This was a good learning experience for testing hardware capabilities and should be done whenever introducing any new aspects to a project.

=== Ball Collision Logic ===
The initial version of the ball collision used pixel values to see if the ball was entering the brick. The problem was that this logic did not account for the case where the ball enters the brick diagonally and the logic for that version of the collision had to be scrapped. The later iteration of the logic was changed to checking whether the ball was potentially crossing into an area that had a brick. This was done by separating the checks into a grid that represented the locations of the blocks. In this case, there was an initial check to see if the ball was crossing a boundary where there might be a new brick. If the ball is crossing the x-boundary and not the y-boundary, then there would be a collision check to see if it collided with a block. If it is crossing the y-boundary and not the x-boundary, once again then there would be a collision check to see if it collided with a block. If it is crossing both the x-boundary and y-boundary, then there would be a check to see which boundary the ball would cross first. This new logic allowed for us to account for the case where the block enters the block diagonally. This taught us that sometimes it is beneficial to scrap a previous iteration and use a new approach to solving a problem because it may lead to better results.

=== Paddle Movement ===
One issue we discovered was the paddle would slow down as it approached the walls. This was due to the difference in "current desired paddle position" and the previous paddle position becoming smaller as the walls were approached, leading to a smaller paddle velocity. To account for this, whenever the accelerometer sent data that mapped to the minimum or maximum paddle index, we would increase the distance of the "current desired paddle position" so the paddle velocity would not slow the paddle down as it approached the walls.

=== Paddle Collision Logic ===
During final integration and testing, one of the biggest issues we encountered was a bug where the ball would not detect a collision with the paddle. The ball would phase right through the paddle instead of bouncing off of it. One of the great ideas that we thought of was to change the color of the ball during each check to see if the checks would go through. This debug check allowed us to verify that section of code that checked for paddle collisions was being skipped even though the ball's next Y position had crossed the paddle's Y position. By having this debugging method, we were able to root cause the issue.

== Conclusion ==
Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

=== Project Video ===
[https://www.youtube.com/watch?v=ipngAGlN86c SJSU CMPE 244 FALL 2021: Embedded Software Project Gameplay]

=== Project Source Code ===
* [https://gitlab.com/fayek.wahhab/sjtwo-c/-/tree/feature/i2s_dma2 Treasure Dive Baseline for Demo Source Code]
* [https://gitlab.com/hobriandh/sjtwo-c/-/tree/final_controller Controller for Demo Source Code]

== References ==
=== Acknowledgement ===
We would like to acknowledge and thank our instructor Preetpal Kang for sharing his knowledge in embedded software development using FreeRTOS.

=== Appendix ===
*[https://gitlab.com/sjtwo-c-dev/sjtwo-c/-/blob/master/LPC408X_7X%20User%20Manual.pdf LPC408x/407x Datasheet]
*[https://www.freertos.org/Documentation/RTOS_book.html FreeRTOS Documentation]
*[http://elm-chan.org/fsw/ff/00index_e.html FatFs - Generic FAT Filesystem Module]
*[https://www.digi.com/resources/library/data-sheets/ds_xbeemultipointmodules Xbee Datasheet]
*[http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems Embedded Drivers & Real Time Operating Systems Lessons]

F21: Treasure Dive

2021-12-18T11:57:56Z

Proj user1: /* Team Members & Responsibilities */

== '''Treasure Dive''' ==
[[File:mermaid.png|thumb|500px|right|Game Screen Capture]]
[[File:Cmpe-244-controller-pic.png|thumb|600px|right|Game Controller]]

== Abstract ==
Treasure Dive is a modern one player game inspired by the popular arcade game Breakout, which was released by Atari in 1976. The ancients rambled about an unconfirmed tale of a shipwreck that saw no survivors. As an adventurous and skilled free diver, you are seeking treasure buried deep in an ocean abyss. You dove down all the way to the seafloor and found the buried treasure but need to make it back to the surface to see another day! The player uses a wireless controller to control an on-screen paddle to break bricks on the top of the screen. When enough bricks are broken, a passage is revealed, allowing you to get closer to the surface.

== Objectives & Introduction ==
The objective of this project is to interface an LPC4088 to a VGA monitor by leveraging the LCD controller and use an accelerometer as the sensor for the player to control the paddle.
Two Zigbees RF modules will be integrated for wireless communication between the controller and LPC4088 board via UART protocol.
Digital audio output will be enabled by developing an I2S driver and porting an audio tracker library.
The General Purpose Direct Memory Access (GPDMA) controller memory-to-memory function will be utilized to decrease memory write times when updating the video buffer.
The DMA controller's memory-to-peripheral function will help reduce the frequent I2S interrupts when loading digital audio contents.
An SD card peripheral driver to facilitate memory exchange between an SD card will also be implemented.

=== Team Members & Responsibilities ===
* [[File:Cmpe-244-brian-pic.png Brian Ho]]
** Wiki project schedule planning
** Interfacing the accelerometer
** Digital to Analog Conversion resistor ladder design used to interface digital LCD controller signal with analog VGA port signal
** PCB planning and acquisition
** Procurement of controller hardware
** Wireless communication via Zigbees RF modules and UART protocol
** Wireless controller logic and integration
** Ball movement and wall/brick collision logic
** Hardware/software development and integration
** Final testing and integration
* [https://www.linkedin.com/in/william-lai-525681139 Billy Lai]
** Wiki project schedule planning
** Increased CPU clock rate from 96 MHz to 120 MHz
** Initial memory write timing and measurements to estimate memory write capacity
** DMA driver memory-to-memory implementation for transferring graphics data to video buffer
** I2S with DMA memory-to-peripheral implementation to reduce memory-write interrupts when I2S Tx FIFO level is low
** Paddle movement and paddle collision logic, testing and integration
** Software development and integration
** Final testing and integration
* Jasdip Sekhon
** Wiki project schedule planning
** PCB design and planning
** File loading method algorithm for storing loaded files in memory
** Audio track and audio effects selection for gameplay audio
** Palette manipulation for gameplay
** Software development and integration
** Final testing and integration
* Isaac Wahhab
** Wiki project schedule planning
** DAC resistor ladder design and integration
** Hardware procurement (LPC4088 board, Zigbees, VGA monitor, etc)
** VGA driving through LCD controller
** Audio tracker library port
** Digital audio output using I2S
** Palette scrolling feature implementation
** Graphics rendering
** Vertical scrolling implementation for the video buffer
** Sprite testing and implementation for the ball
** SD card peripheral driver for reading assets from SD card
** Art assets acquisition
** Hardware/software development and integration
** Gameplay logic design
** Gameplay audio and level design
** Final testing and integration

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
*09/21/2021
|
*09/27/2021
|
* Read previous projects, gather information and discuss among the group members.
* Initial ordering of parts (LPC 4088 board, 64x64 LED screen)
|
* Completed
* Completed
|-
! scope="row"| 2
|
*09/28/2021
|
*10/04/2021
|
* Submit project proposals
|
* Completed
|-
! scope="row"| 3
|
*10/05/2021
|
*10/11/2021
|
* Test if board can drive VGA
* Test varying clock rates
* Hardware cursor on VGA
* Rough display driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 4
|
* 10/12/2021
|
* 10/18/2021
|
* DAC resistor ladder
* RGB channels
* Palette shifting
* Measure memory buffer writing timing
* Wiki schedule planning
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
*10/19/2021
|
*10/25/2021
|
* Calculate upper-bound for VGA refresh timings
* Create GitLab repository for project
* Joystick functionality
* SD card driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
*10/26/2021
|
*11/01/2021
|
* Audio peripheral
* Wireless communication using Zigbee module
* PCB planning
* Accelerometer driver
* Continue SD card driver
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/02/2021

|
* 11/08/2021
|
* Continue audio peripheral
* Continue wireless communication using Zigbee module
* Continue accelerometer driver
* PCB finalizing and ordering
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 11/09/2021
|
* 11/15/2021
|
* Convert PNG data to graphics format
* Game background functionality
* Sprite functionality
* User interface
* Integrate components
* Initial testing
|
* Completed
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 11/16/2021
|
* 11/22/2021
|
* Continue everything from last week
* Game physics (wall collision and AI movement)
* Measurements for wooden case
* Art assets (audio and visual)
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 10
|
*11/23/2021
|
*11/29/2021
|

* Fine-tune user experience
* Make case
|
* Completed
* Completed
|-
! scope="row"| 11
|
* 11/30/2021
|
* 12/06/2021
|
* Continue fine-tune user experience
* Integrate hardware
|
* Completed
* Completed
|-
! scope="row"| 12
|
* 12/07/2021
|
* 12/16/2021
|
* Final testing
* Final demo
|
* Completed
* Completed
|-
|}

== Bill Of Materials ==
{| class="wikitable" style="margin-left: 0px; margin-right: auto;"
|-
! scope="col"| Item#
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| [https://www.embeddedartists.com/products/lpc4088-developers-kit LPC4088-32 Developer's Kit]
| 1
| $300.00
|-
! scope="row"| 2
| Microcontroller Board
| [http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems/page/sj2-board SJ2 Board]
| 1
| $50.00
|-
! scope="row"| 3
| Bluetooth Module
| [https://hub.digi.com/support/products/digi-xbee/digi-xbee-s1 Digi XBee-S1]
| 2
| N/A
|-
! scope="row"| 4
| DAC PCB
| [https://jlcpcb.com JLCPCB (Set of 5)]
| 1
| $15.00
|-
! scope="row"| 4
| Monitor
| Asus TFT Monitor
| 1
| N/A
|-
! scope="row"| 5
| Mini Breadboard
| [https://www.amazon.com/EL-CK-002-Electronic-Breadboard-Capacitor-Potentiometer/dp/B01ERP6WL4/ref=pd_sbs_147_2/132-6936347-2117569?_encoding=UTF8&pd_rd_i=B01ERP6WL4&pd_rd_r=6a783095-ec87-42ae-9f01-a5ebc9a44607&pd_rd_w=V2aU6&pd_rd_wg=11h1i&pf_rd_p=d66372fe-68a6-48a3-90ec-41d7f64212be&pf_rd_r=368WNKXFBBQY2FJQ4119&psc=1&refRID=368WNKXFBBQY2FJQ4119 ELEGOO Mini Breadboard]
| 1
| $13.99
|-
! scope="row"| 6
| Arcade buttons
| [https://www.amazon.com/EG-STARTS-Illuminated-Buttons-Raspberry/dp/B01N11BDX9?pd_rd_w=DiSpt&pf_rd_p=5bc65336-a8b0-4888-9e70-4d4ce2f6bd59&pf_rd_r=JCH2VQYXFPHND4XZ4FAM&pd_rd_r=8d6243dd-e1bd-4dae-9b19-be2e00f3085f&pd_rd_wg=K4lt9&pd_rd_i=B01N11BDX9&psc=1&ref_=pd_bap_d_rp_1_i EG STARTS LED Arcade Buttons]
| 2
| $11.99
|-
! scope="row"| 7
| VGA cable
| N/A
| 1
| N/A
|-
! scope="row"| 7
| SD card
| N/A
| 1
| N/A
|-
|}

== Design & Implementation ==

=== Hardware Design ===
We designed our PCB using Autodesk EAGLE based on the tutorial on PCB design during the class lecture. We ordered our PCB to be manufactured from JLCPCB. The PCB consists of an R-2R resistor ladder DAC for the purpose of converting the digital signal from the microcontroller to analog to display graphics on the VGA monitor. Since the VGA input for each color channel is 0-0.7V, the resistor ladder scaled the LPC4088 outputs into proper DAC values for the VGA to take in. Five bits are passed to each of the red, green, and blue channels of the VGA in addition to other signals, such as Horizontal sync (HSYNC) and Vertical sync (VSYNC).
 
[[File:pcbSchematic.png|500px|thumb|left|PCB Schematic]]
[[File:pcbLayout.png|500px|thumb|right|PCB Layout]]
[[File:pcbPic-min.png|500px|thumb|right|Manufactured PCB]]
[[File:pcbManufactured.png|500px|thumb|left|Soldered PCB]]
 

=== Hardware Interface ===

Two Digi XBee devices were used to communicate between the SJ2-Board controller and the LPC4088 board to control the paddle on the game. The XBee devices were programmed using the XTCU programming application provided by Digi International. The SJ2-Board used one XBee as a transmitted and the LPC4088 used another as a receiver. The two modules communicated using UART frames and a single byte represented any change of movement or button presses sent by the controller.

{| class="wikitable"
|-
! scope="col"| XCTU
! scope="col"| Transmitter
! scope="col"| Receiver
|-
! scope="row"| CH
| C
| C
|-
! scope="row"| ID
| 6969
| 6969
|-
! scope="row"| CE
| Coordinator
| Endpoint
|-
! scope="row"| Baud Rate
| 9600
| 9600
|-
|}
[[File:sj2Board.png|380px|thumb|center|SJ2 Board]]
[[File:CMPE146_F16_MW_XbeeModule.jpg|200px|thumb|center|XBee Device]]
 

Two LED buttons were used to interface with the controller as inputs. They were both set up using GPIO.
[[File:VGA LED Button.jpg|200px|thumb|center|LED Button]]
 
[[File:VGA_Controller_Block_Diagram.PNG|380px|thumb|center|Controller Block Diagram]]
 

==== I2S Peripheral and Stereo Audio Decoder ====

The LPC4088 is equipped with an I2S stereo encoder. This module provides a hardware-enabled solution for sending I2S-encoded audio data packets to stereo decoders, which produce the corresponding audio waveform. For this project, since 16-bit stereo audio playback was chosen, data packets transmitted on this interface were sent in the following format:

[[File:I2S_audio_data_word.png]]

Here, each word of data encodes a single packet of audio in two 16-bit sections: one signal level for the left channel, and another for the right. These 4 bytes are sent at a rate of 44100 Hz, resulting in a total data transmission of 176400 bytes a second. Rather than needing to be software-defined, this transmission is automatically handled by hardware using a 8-word deep FIFO queue that manages streaming out the audio data at a fixed rate, saving a significant amount of CPU cycles.

This encoded audio signal was sent to a dedicated external [https://www.nxp.com/docs/en/data-sheet/UDA1380.pdf stereo decoder], which output to a 3.5 mm audio jack connected to speakers in the VGA monitor.

==== LCD Peripheral ====

The LCD peripheral was configured for 16-bit color palette (5 bits of color data for each color level -- red, green, and blue -- and an unused intensity bit), at a resolution of 800 by 600 pixels, and with an active color palette size of 256 colors. Since the logic for driving an LCD display has the same concepts (horizontal blanking, vertical blanking, etc) and timing information for any given resolution was the same, we were able to directly drive a VGA display with the same signals and timing data (after the digital signal had been converted to analog).

[[File:LCD_%2B_VGA_Video_signal.png]]

The timing register configurations needed to achieve the requested video display was derived from a dedicated website for [http://tinyvga.com/vga-timing/800x600@60Hz VGA signal info].

==== External Memory ====

The game's visual assets and tracker audio file (both after being loaded from an SD card) and the game's video buffer required more memory than what is present natively in the microcontroller. To accommodate this, an external 32MiB memory module was used. This module was interfaced with using the external memory controller peripheral present on the board.

=== Software Design ===
==== Controller ====

There is an internal accelerometer on the SJ2 board that is utilized as the movement controls for the paddle movement. The y-axis value of the magnitude is polled 17 times in 17 milliseconds and is divided by 17 to get an average reading per that time frame. The maximum tilt value that can be read by the controller is set to 700 out of the total 1000 (highest possible value). The current tilt value is divided by 31 to get a total of 32 possible readings to represent 5 bits of magnitude.

The controller sends one byte of data to the console whenever there is any change in magnitude on the accelerometer or button changes. The byte contains all the information needed to control the paddle on the screen. The first two bits on the byte represent 0 if there is no button input and 1 if there is a button input. The third bit represents whether the controller is tilted left or right with 0 being left and 1 being right. The last five bits represent the controller accelerometer's magnitude.

[[File:VGA_UART_Frame.PNG]]

==== Audio ====

Sound design was provided to the project (in the form of music and sound effects) through the use of a [https://https://en.wikipedia.org/wiki/Music_tracker tracker audio library]. Source code was adopted from the [https://github.com/schismtracker/schismtracker Schism Tracker] library and altered with the following modifications:

- All UI logic was stripped from the source code

- All file handling logic was reworked to use the SD card reading functions, as the project does not support POSIX file calls

- The [https://www.libsdl.org/ Simple DirectMedia Layer] hooks were replaced with a FreeRTOS task that periodically called the waveform generation callback function, which was passed to the I2S peripheral hooked up to a stereo decoder.

=== Implementation ===

==== Ball Logic ====

The ball is controlled by the buttons and the controller accelerometer values. When the button is pressed, the ball is initially sent with a random velocity. The velocity is determined by a substate timer and is initially set as one of 6 possible values.

{| class="wikitable"
|-
! scope="col"| Substate Count
! scope="col"| X-Velocity
! scope="col"| Y-Velocity
|-
! scope="row"| 0
| 4
| -2
|-
! scope="row"| 1
| -4
| -2
|-
! scope="row"| 2
| 2
| -4
|-
! scope="row"| 3
| -2
| -4
|-
! scope="row"| 4
| 1
| -5
|-
! scope="row"| 5
| -1
| -5
|-
|}

The ball's velocity changes whenever it detects a collision with either the side of the screen, the bricks on the top of the level, or the paddle at the bottom. If colliding with the paddle, the ball's velocity will take into account the paddle's velocity and add it to its current velocity. This is done by adding the paddle's x-velocity and adjusting the y-velocity by normalizing the velocity vectors to have a magnitude of 5. This means the ball will always be moving with a magnitude of 5 in any direction after it hits the paddle. If the ball collides with anything other than the paddle, then the velocity of the ball is changed with no increase or decrease of magnitude. If the ball collides on the side of a wall or block, the x-velocity will reflect (go from negative to positive or vice-versa) and the y-velocity will remain constant. If the ball collides on the top or bottom of a block, the y-velocity will reflect and the x-velocity will remain constant. If it appears that the ball collides with the side and top or bottom of something, then the ball will check whichever face it will collide with first and then calculate the other face. Lastly, when the ball collides with the bottom of the screen, then the player experiences a "death" and the ball will reset on the paddle. If the ball collides with the top of the screen, the screen will scroll up for the user to play on the next level.

Whenever the ball collides with a brick, the brick will disappear. To accomplish this, there is a 2-D array of unsigned integers that contains the values of the bricks. If the integer is 0, then there is no block. If the integer is any value other than 0, then the block is filled and has collision logic. If the ball approaches a region that might have a block, then there will be a check within that 2-D to see if the block is filled or empty. If the block is filled, there will be the collision logic that changes the vector of the ball and sets the integer value in the 2-D array to 0 to represent that the block has been broken.

==== Paddle Movement ====
The paddle movement is controlled by the accelerometer data being wirelessly transmitted from the SJ-Two board to the LPC4088 board. The accelerometer data will map to 63 possible indexes for the paddle location. Originally, the paddle would jump directly into the corresponding index location depending on the accelerometer input. We later iterated on this process by implementing an algorithm to smoothly transition into these indexes. This was done by calculating a paddle velocity using the "current desired paddle position" and the previous paddle position. We adjusted the parameters of the algorithm iteratively to produce the smooth paddle movement.

==== Paddle Collision Logic ====
The position of the ball is calculated by adding delta_x and delta_y to the ball's current x and y position. For each frame, if the ball's next Y position crosses the paddle's Y position, we would check for a paddle collision. To do this, we used the delta_x and delta_y to get the ball's movement as a slope. We then substituted the slope, the ball's current position, and 1 pixel above the paddle's Y position into a point-slope equation (y2 - y1) = m(x2 - x1) to get the paddle's X position at the time of the ball reaching the pixel above paddle's Y position. After finding the location where the ball is expected to collide with the paddle, we then calculated the ball's collision magnitude and the ball's next location's magnitude. Using a ratio of these two magnitudes, we have a frame time fraction of when the ball possibly collided with the paddle. Then, by using the paddle's current position and paddle velocity, we are able to find where the paddle previously was, and measure how far the paddle had moved during that time frame. By multiplying the time frame from the ball's collision magnitude to the paddle's change in X position, we are able to find whether the paddle was under the ball at the time of the ball reaching the pixel above the paddle's Y position.

== Testing & Technical Challenges ==

=== VGA Interfacing ===
A DAC resistor ladder was designed and a corresponding custom PCB was acquired to interface the 3.3V digital LCD controller signal with the analog VGA input signal. The VGA cable takes between 0V (completely dark) and 0.7V (maximum brightness) for color pin inputs. The DAC resistor ladder was designed and implemented on a breadboard to verify the design. Validating the design on a breadboard gave us a clear vision during PCB design and planning.

=== VGA Write Timings ===
There were initial tests that were required to be done to see if using a VGA monitor as an output was a viable option for this project. We had to test memory write timings to see if they were fast enough to match the refresh rate of the monitor. Those tests would decide how much of the screen buffer we could write to during the blanking periods during the VGA refresh. This was a good learning experience for testing hardware capabilities and should be done whenever introducing any new aspects to a project.

=== Ball Collision Logic ===
The initial version of the ball collision used pixel values to see if the ball was entering the brick. The problem was that this logic did not account for the case where the ball enters the brick diagonally and the logic for that version of the collision had to be scrapped. The later iteration of the logic was changed to checking whether the ball was potentially crossing into an area that had a brick. This was done by separating the checks into a grid that represented the locations of the blocks. In this case, there was an initial check to see if the ball was crossing a boundary where there might be a new brick. If the ball is crossing the x-boundary and not the y-boundary, then there would be a collision check to see if it collided with a block. If it is crossing the y-boundary and not the x-boundary, once again then there would be a collision check to see if it collided with a block. If it is crossing both the x-boundary and y-boundary, then there would be a check to see which boundary the ball would cross first. This new logic allowed for us to account for the case where the block enters the block diagonally. This taught us that sometimes it is beneficial to scrap a previous iteration and use a new approach to solving a problem because it may lead to better results.

=== Paddle Movement ===
One issue we discovered was the paddle would slow down as it approached the walls. This was due to the difference in "current desired paddle position" and the previous paddle position becoming smaller as the walls were approached, leading to a smaller paddle velocity. To account for this, whenever the accelerometer sent data that mapped to the minimum or maximum paddle index, we would increase the distance of the "current desired paddle position" so the paddle velocity would not slow the paddle down as it approached the walls.

=== Paddle Collision Logic ===
During final integration and testing, one of the biggest issues we encountered was a bug where the ball would not detect a collision with the paddle. The ball would phase right through the paddle instead of bouncing off of it. One of the great ideas that we thought of was to change the color of the ball during each check to see if the checks would go through. This debug check allowed us to verify that section of code that checked for paddle collisions was being skipped even though the ball's next Y position had crossed the paddle's Y position. By having this debugging method, we were able to root cause the issue.

== Conclusion ==
Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

=== Project Video ===
[https://www.youtube.com/watch?v=ipngAGlN86c SJSU CMPE 244 FALL 2021: Embedded Software Project Gameplay]

=== Project Source Code ===
* [https://gitlab.com/fayek.wahhab/sjtwo-c/-/tree/feature/i2s_dma2 Treasure Dive Baseline for Demo Source Code]
* [https://gitlab.com/hobriandh/sjtwo-c/-/tree/final_controller Controller for Demo Source Code]

== References ==
=== Acknowledgement ===
We would like to acknowledge and thank our instructor Preetpal Kang for sharing his knowledge in embedded software development using FreeRTOS.

=== Appendix ===
*[https://gitlab.com/sjtwo-c-dev/sjtwo-c/-/blob/master/LPC408X_7X%20User%20Manual.pdf LPC408x/407x Datasheet]
*[https://www.freertos.org/Documentation/RTOS_book.html FreeRTOS Documentation]
*[http://elm-chan.org/fsw/ff/00index_e.html FatFs - Generic FAT Filesystem Module]
*[https://www.digi.com/resources/library/data-sheets/ds_xbeemultipointmodules Xbee Datasheet]
*[http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems Embedded Drivers & Real Time Operating Systems Lessons]

File:Cmpe-244-brian-pic.png

2021-12-18T11:57:22Z

Proj user1: Proj user1 uploaded a new version of File:Cmpe-244-brian-pic.png

Brian Ho

File:Cmpe-244-brian-pic.png

2021-12-18T11:56:06Z

Proj user1: Brian Ho

Brian Ho

F21: Treasure Dive

2021-12-18T11:51:19Z

Proj user1:

== '''Treasure Dive''' ==
[[File:mermaid.png|thumb|500px|right|Game Screen Capture]]
[[File:Cmpe-244-controller-pic.png|thumb|600px|right|Game Controller]]

== Abstract ==
Treasure Dive is a modern one player game inspired by the popular arcade game Breakout, which was released by Atari in 1976. The ancients rambled about an unconfirmed tale of a shipwreck that saw no survivors. As an adventurous and skilled free diver, you are seeking treasure buried deep in an ocean abyss. You dove down all the way to the seafloor and found the buried treasure but need to make it back to the surface to see another day! The player uses a wireless controller to control an on-screen paddle to break bricks on the top of the screen. When enough bricks are broken, a passage is revealed, allowing you to get closer to the surface.

== Objectives & Introduction ==
The objective of this project is to interface an LPC4088 to a VGA monitor by leveraging the LCD controller and use an accelerometer as the sensor for the player to control the paddle.
Two Zigbees RF modules will be integrated for wireless communication between the controller and LPC4088 board via UART protocol.
Digital audio output will be enabled by developing an I2S driver and porting an audio tracker library.
The General Purpose Direct Memory Access (GPDMA) controller memory-to-memory function will be utilized to decrease memory write times when updating the video buffer.
The DMA controller's memory-to-peripheral function will help reduce the frequent I2S interrupts when loading digital audio contents.
An SD card peripheral driver to facilitate memory exchange between an SD card will also be implemented.

=== Team Members & Responsibilities ===
* Brian Ho
** Wiki project schedule planning
** Interfacing the accelerometer
** Digital to Analog Conversion resistor ladder design used to interface digital LCD controller signal with analog VGA port signal
** PCB planning and acquisition
** Procurement of controller hardware
** Wireless communication via Zigbees RF modules and UART protocol
** Wireless controller logic and integration
** Ball movement and wall/brick collision logic
** Hardware/software development and integration
** Final testing and integration
* [https://www.linkedin.com/in/william-lai-525681139 Billy Lai]
** Wiki project schedule planning
** Increased CPU clock rate from 96 MHz to 120 MHz
** Initial memory write timing and measurements to estimate memory write capacity
** DMA driver memory-to-memory implementation for transferring graphics data to video buffer
** I2S with DMA memory-to-peripheral implementation to reduce memory-write interrupts when I2S Tx FIFO level is low
** Paddle movement and paddle collision logic, testing and integration
** Software development and integration
** Final testing and integration
* Jasdip Sekhon
** Wiki project schedule planning
** PCB design and planning
** File loading method algorithm for storing loaded files in memory
** Audio track and audio effects selection for gameplay audio
** Palette manipulation for gameplay
** Software development and integration
** Final testing and integration
* Isaac Wahhab
** Wiki project schedule planning
** DAC resistor ladder design and integration
** Hardware procurement (LPC4088 board, Zigbees, VGA monitor, etc)
** VGA driving through LCD controller
** Audio tracker library port
** Digital audio output using I2S
** Palette scrolling feature implementation
** Graphics rendering
** Vertical scrolling implementation for the video buffer
** Sprite testing and implementation for the ball
** SD card peripheral driver for reading assets from SD card
** Art assets acquisition
** Hardware/software development and integration
** Gameplay logic design
** Gameplay audio and level design
** Final testing and integration

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
*09/21/2021
|
*09/27/2021
|
* Read previous projects, gather information and discuss among the group members.
* Initial ordering of parts (LPC 4088 board, 64x64 LED screen)
|
* Completed
* Completed
|-
! scope="row"| 2
|
*09/28/2021
|
*10/04/2021
|
* Submit project proposals
|
* Completed
|-
! scope="row"| 3
|
*10/05/2021
|
*10/11/2021
|
* Test if board can drive VGA
* Test varying clock rates
* Hardware cursor on VGA
* Rough display driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 4
|
* 10/12/2021
|
* 10/18/2021
|
* DAC resistor ladder
* RGB channels
* Palette shifting
* Measure memory buffer writing timing
* Wiki schedule planning
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
*10/19/2021
|
*10/25/2021
|
* Calculate upper-bound for VGA refresh timings
* Create GitLab repository for project
* Joystick functionality
* SD card driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
*10/26/2021
|
*11/01/2021
|
* Audio peripheral
* Wireless communication using Zigbee module
* PCB planning
* Accelerometer driver
* Continue SD card driver
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/02/2021

|
* 11/08/2021
|
* Continue audio peripheral
* Continue wireless communication using Zigbee module
* Continue accelerometer driver
* PCB finalizing and ordering
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 11/09/2021
|
* 11/15/2021
|
* Convert PNG data to graphics format
* Game background functionality
* Sprite functionality
* User interface
* Integrate components
* Initial testing
|
* Completed
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 11/16/2021
|
* 11/22/2021
|
* Continue everything from last week
* Game physics (wall collision and AI movement)
* Measurements for wooden case
* Art assets (audio and visual)
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 10
|
*11/23/2021
|
*11/29/2021
|

* Fine-tune user experience
* Make case
|
* Completed
* Completed
|-
! scope="row"| 11
|
* 11/30/2021
|
* 12/06/2021
|
* Continue fine-tune user experience
* Integrate hardware
|
* Completed
* Completed
|-
! scope="row"| 12
|
* 12/07/2021
|
* 12/16/2021
|
* Final testing
* Final demo
|
* Completed
* Completed
|-
|}

== Bill Of Materials ==
{| class="wikitable" style="margin-left: 0px; margin-right: auto;"
|-
! scope="col"| Item#
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| [https://www.embeddedartists.com/products/lpc4088-developers-kit LPC4088-32 Developer's Kit]
| 1
| $300.00
|-
! scope="row"| 2
| Microcontroller Board
| [http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems/page/sj2-board SJ2 Board]
| 1
| $50.00
|-
! scope="row"| 3
| Bluetooth Module
| [https://hub.digi.com/support/products/digi-xbee/digi-xbee-s1 Digi XBee-S1]
| 2
| N/A
|-
! scope="row"| 4
| DAC PCB
| [https://jlcpcb.com JLCPCB (Set of 5)]
| 1
| $15.00
|-
! scope="row"| 4
| Monitor
| Asus TFT Monitor
| 1
| N/A
|-
! scope="row"| 5
| Mini Breadboard
| [https://www.amazon.com/EL-CK-002-Electronic-Breadboard-Capacitor-Potentiometer/dp/B01ERP6WL4/ref=pd_sbs_147_2/132-6936347-2117569?_encoding=UTF8&pd_rd_i=B01ERP6WL4&pd_rd_r=6a783095-ec87-42ae-9f01-a5ebc9a44607&pd_rd_w=V2aU6&pd_rd_wg=11h1i&pf_rd_p=d66372fe-68a6-48a3-90ec-41d7f64212be&pf_rd_r=368WNKXFBBQY2FJQ4119&psc=1&refRID=368WNKXFBBQY2FJQ4119 ELEGOO Mini Breadboard]
| 1
| $13.99
|-
! scope="row"| 6
| Arcade buttons
| [https://www.amazon.com/EG-STARTS-Illuminated-Buttons-Raspberry/dp/B01N11BDX9?pd_rd_w=DiSpt&pf_rd_p=5bc65336-a8b0-4888-9e70-4d4ce2f6bd59&pf_rd_r=JCH2VQYXFPHND4XZ4FAM&pd_rd_r=8d6243dd-e1bd-4dae-9b19-be2e00f3085f&pd_rd_wg=K4lt9&pd_rd_i=B01N11BDX9&psc=1&ref_=pd_bap_d_rp_1_i EG STARTS LED Arcade Buttons]
| 2
| $11.99
|-
! scope="row"| 7
| VGA cable
| N/A
| 1
| N/A
|-
! scope="row"| 7
| SD card
| N/A
| 1
| N/A
|-
|}

== Design & Implementation ==

=== Hardware Design ===
We designed our PCB using Autodesk EAGLE based on the tutorial on PCB design during the class lecture. We ordered our PCB to be manufactured from JLCPCB. The PCB consists of an R-2R resistor ladder DAC for the purpose of converting the digital signal from the microcontroller to analog to display graphics on the VGA monitor. Since the VGA input for each color channel is 0-0.7V, the resistor ladder scaled the LPC4088 outputs into proper DAC values for the VGA to take in. Five bits are passed to each of the red, green, and blue channels of the VGA in addition to other signals, such as Horizontal sync (HSYNC) and Vertical sync (VSYNC).
 
[[File:pcbSchematic.png|500px|thumb|left|PCB Schematic]]
[[File:pcbLayout.png|500px|thumb|right|PCB Layout]]
[[File:pcbPic-min.png|500px|thumb|right|Manufactured PCB]]
[[File:pcbManufactured.png|500px|thumb|left|Soldered PCB]]
 

=== Hardware Interface ===

Two Digi XBee devices were used to communicate between the SJ2-Board controller and the LPC4088 board to control the paddle on the game. The XBee devices were programmed using the XTCU programming application provided by Digi International. The SJ2-Board used one XBee as a transmitted and the LPC4088 used another as a receiver. The two modules communicated using UART frames and a single byte represented any change of movement or button presses sent by the controller.

{| class="wikitable"
|-
! scope="col"| XCTU
! scope="col"| Transmitter
! scope="col"| Receiver
|-
! scope="row"| CH
| C
| C
|-
! scope="row"| ID
| 6969
| 6969
|-
! scope="row"| CE
| Coordinator
| Endpoint
|-
! scope="row"| Baud Rate
| 9600
| 9600
|-
|}
[[File:sj2Board.png|380px|thumb|center|SJ2 Board]]
[[File:CMPE146_F16_MW_XbeeModule.jpg|200px|thumb|center|XBee Device]]
 

Two LED buttons were used to interface with the controller as inputs. They were both set up using GPIO.
[[File:VGA LED Button.jpg|200px|thumb|center|LED Button]]
 
[[File:VGA_Controller_Block_Diagram.PNG|380px|thumb|center|Controller Block Diagram]]
 

==== I2S Peripheral and Stereo Audio Decoder ====

The LPC4088 is equipped with an I2S stereo encoder. This module provides a hardware-enabled solution for sending I2S-encoded audio data packets to stereo decoders, which produce the corresponding audio waveform. For this project, since 16-bit stereo audio playback was chosen, data packets transmitted on this interface were sent in the following format:

[[File:I2S_audio_data_word.png]]

Here, each word of data encodes a single packet of audio in two 16-bit sections: one signal level for the left channel, and another for the right. These 4 bytes are sent at a rate of 44100 Hz, resulting in a total data transmission of 176400 bytes a second. Rather than needing to be software-defined, this transmission is automatically handled by hardware using a 8-word deep FIFO queue that manages streaming out the audio data at a fixed rate, saving a significant amount of CPU cycles.

This encoded audio signal was sent to a dedicated external [https://www.nxp.com/docs/en/data-sheet/UDA1380.pdf stereo decoder], which output to a 3.5 mm audio jack connected to speakers in the VGA monitor.

==== LCD Peripheral ====

The LCD peripheral was configured for 16-bit color palette (5 bits of color data for each color level -- red, green, and blue -- and an unused intensity bit), at a resolution of 800 by 600 pixels, and with an active color palette size of 256 colors. Since the logic for driving an LCD display has the same concepts (horizontal blanking, vertical blanking, etc) and timing information for any given resolution was the same, we were able to directly drive a VGA display with the same signals and timing data (after the digital signal had been converted to analog).

[[File:LCD_%2B_VGA_Video_signal.png]]

The timing register configurations needed to achieve the requested video display was derived from a dedicated website for [http://tinyvga.com/vga-timing/800x600@60Hz VGA signal info].

==== External Memory ====

The game's visual assets and tracker audio file (both after being loaded from an SD card) and the game's video buffer required more memory than what is present natively in the microcontroller. To accommodate this, an external 32MiB memory module was used. This module was interfaced with using the external memory controller peripheral present on the board.

=== Software Design ===
==== Controller ====

There is an internal accelerometer on the SJ2 board that is utilized as the movement controls for the paddle movement. The y-axis value of the magnitude is polled 17 times in 17 milliseconds and is divided by 17 to get an average reading per that time frame. The maximum tilt value that can be read by the controller is set to 700 out of the total 1000 (highest possible value). The current tilt value is divided by 31 to get a total of 32 possible readings to represent 5 bits of magnitude.

The controller sends one byte of data to the console whenever there is any change in magnitude on the accelerometer or button changes. The byte contains all the information needed to control the paddle on the screen. The first two bits on the byte represent 0 if there is no button input and 1 if there is a button input. The third bit represents whether the controller is tilted left or right with 0 being left and 1 being right. The last five bits represent the controller accelerometer's magnitude.

[[File:VGA_UART_Frame.PNG]]

==== Audio ====

Sound design was provided to the project (in the form of music and sound effects) through the use of a [https://https://en.wikipedia.org/wiki/Music_tracker tracker audio library]. Source code was adopted from the [https://github.com/schismtracker/schismtracker Schism Tracker] library and altered with the following modifications:

- All UI logic was stripped from the source code

- All file handling logic was reworked to use the SD card reading functions, as the project does not support POSIX file calls

- The [https://www.libsdl.org/ Simple DirectMedia Layer] hooks were replaced with a FreeRTOS task that periodically called the waveform generation callback function, which was passed to the I2S peripheral hooked up to a stereo decoder.

=== Implementation ===

==== Ball Logic ====

The ball is controlled by the buttons and the controller accelerometer values. When the button is pressed, the ball is initially sent with a random velocity. The velocity is determined by a substate timer and is initially set as one of 6 possible values.

{| class="wikitable"
|-
! scope="col"| Substate Count
! scope="col"| X-Velocity
! scope="col"| Y-Velocity
|-
! scope="row"| 0
| 4
| -2
|-
! scope="row"| 1
| -4
| -2
|-
! scope="row"| 2
| 2
| -4
|-
! scope="row"| 3
| -2
| -4
|-
! scope="row"| 4
| 1
| -5
|-
! scope="row"| 5
| -1
| -5
|-
|}

The ball's velocity changes whenever it detects a collision with either the side of the screen, the bricks on the top of the level, or the paddle at the bottom. If colliding with the paddle, the ball's velocity will take into account the paddle's velocity and add it to its current velocity. This is done by adding the paddle's x-velocity and adjusting the y-velocity by normalizing the velocity vectors to have a magnitude of 5. This means the ball will always be moving with a magnitude of 5 in any direction after it hits the paddle. If the ball collides with anything other than the paddle, then the velocity of the ball is changed with no increase or decrease of magnitude. If the ball collides on the side of a wall or block, the x-velocity will reflect (go from negative to positive or vice-versa) and the y-velocity will remain constant. If the ball collides on the top or bottom of a block, the y-velocity will reflect and the x-velocity will remain constant. If it appears that the ball collides with the side and top or bottom of something, then the ball will check whichever face it will collide with first and then calculate the other face. Lastly, when the ball collides with the bottom of the screen, then the player experiences a "death" and the ball will reset on the paddle. If the ball collides with the top of the screen, the screen will scroll up for the user to play on the next level.

Whenever the ball collides with a brick, the brick will disappear. To accomplish this, there is a 2-D array of unsigned integers that contains the values of the bricks. If the integer is 0, then there is no block. If the integer is any value other than 0, then the block is filled and has collision logic. If the ball approaches a region that might have a block, then there will be a check within that 2-D to see if the block is filled or empty. If the block is filled, there will be the collision logic that changes the vector of the ball and sets the integer value in the 2-D array to 0 to represent that the block has been broken.

==== Paddle Movement ====
The paddle movement is controlled by the accelerometer data being wirelessly transmitted from the SJ-Two board to the LPC4088 board. The accelerometer data will map to 63 possible indexes for the paddle location. Originally, the paddle would jump directly into the corresponding index location depending on the accelerometer input. We later iterated on this process by implementing an algorithm to smoothly transition into these indexes. This was done by calculating a paddle velocity using the "current desired paddle position" and the previous paddle position. We adjusted the parameters of the algorithm iteratively to produce the smooth paddle movement.

==== Paddle Collision Logic ====
The position of the ball is calculated by adding delta_x and delta_y to the ball's current x and y position. For each frame, if the ball's next Y position crosses the paddle's Y position, we would check for a paddle collision. To do this, we used the delta_x and delta_y to get the ball's movement as a slope. We then substituted the slope, the ball's current position, and 1 pixel above the paddle's Y position into a point-slope equation (y2 - y1) = m(x2 - x1) to get the paddle's X position at the time of the ball reaching the pixel above paddle's Y position. After finding the location where the ball is expected to collide with the paddle, we then calculated the ball's collision magnitude and the ball's next location's magnitude. Using a ratio of these two magnitudes, we have a frame time fraction of when the ball possibly collided with the paddle. Then, by using the paddle's current position and paddle velocity, we are able to find where the paddle previously was, and measure how far the paddle had moved during that time frame. By multiplying the time frame from the ball's collision magnitude to the paddle's change in X position, we are able to find whether the paddle was under the ball at the time of the ball reaching the pixel above the paddle's Y position.

== Testing & Technical Challenges ==

=== VGA Interfacing ===
A DAC resistor ladder was designed and a corresponding custom PCB was acquired to interface the 3.3V digital LCD controller signal with the analog VGA input signal. The VGA cable takes between 0V (completely dark) and 0.7V (maximum brightness) for color pin inputs. The DAC resistor ladder was designed and implemented on a breadboard to verify the design. Validating the design on a breadboard gave us a clear vision during PCB design and planning.

=== VGA Write Timings ===
There were initial tests that were required to be done to see if using a VGA monitor as an output was a viable option for this project. We had to test memory write timings to see if they were fast enough to match the refresh rate of the monitor. Those tests would decide how much of the screen buffer we could write to during the blanking periods during the VGA refresh. This was a good learning experience for testing hardware capabilities and should be done whenever introducing any new aspects to a project.

=== Ball Collision Logic ===
The initial version of the ball collision used pixel values to see if the ball was entering the brick. The problem was that this logic did not account for the case where the ball enters the brick diagonally and the logic for that version of the collision had to be scrapped. The later iteration of the logic was changed to checking whether the ball was potentially crossing into an area that had a brick. This was done by separating the checks into a grid that represented the locations of the blocks. In this case, there was an initial check to see if the ball was crossing a boundary where there might be a new brick. If the ball is crossing the x-boundary and not the y-boundary, then there would be a collision check to see if it collided with a block. If it is crossing the y-boundary and not the x-boundary, once again then there would be a collision check to see if it collided with a block. If it is crossing both the x-boundary and y-boundary, then there would be a check to see which boundary the ball would cross first. This new logic allowed for us to account for the case where the block enters the block diagonally. This taught us that sometimes it is beneficial to scrap a previous iteration and use a new approach to solving a problem because it may lead to better results.

=== Paddle Movement ===
One issue we discovered was the paddle would slow down as it approached the walls. This was due to the difference in "current desired paddle position" and the previous paddle position becoming smaller as the walls were approached, leading to a smaller paddle velocity. To account for this, whenever the accelerometer sent data that mapped to the minimum or maximum paddle index, we would increase the distance of the "current desired paddle position" so the paddle velocity would not slow the paddle down as it approached the walls.

=== Paddle Collision Logic ===
During final integration and testing, one of the biggest issues we encountered was a bug where the ball would not detect a collision with the paddle. The ball would phase right through the paddle instead of bouncing off of it. One of the great ideas that we thought of was to change the color of the ball during each check to see if the checks would go through. This debug check allowed us to verify that section of code that checked for paddle collisions was being skipped even though the ball's next Y position had crossed the paddle's Y position. By having this debugging method, we were able to root cause the issue.

== Conclusion ==
Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

=== Project Video ===
[https://www.youtube.com/watch?v=ipngAGlN86c SJSU CMPE 244 FALL 2021: Embedded Software Project Gameplay]

=== Project Source Code ===
* [https://gitlab.com/fayek.wahhab/sjtwo-c/-/tree/feature/i2s_dma2 Treasure Dive Baseline for Demo Source Code]
* [https://gitlab.com/hobriandh/sjtwo-c/-/tree/final_controller Controller for Demo Source Code]

== References ==
=== Acknowledgement ===
We would like to acknowledge and thank our instructor Preetpal Kang for sharing his knowledge in embedded software development using FreeRTOS.

=== Appendix ===
*[https://gitlab.com/sjtwo-c-dev/sjtwo-c/-/blob/master/LPC408X_7X%20User%20Manual.pdf LPC408x/407x Datasheet]
*[https://www.freertos.org/Documentation/RTOS_book.html FreeRTOS Documentation]
*[http://elm-chan.org/fsw/ff/00index_e.html FatFs - Generic FAT Filesystem Module]
*[https://www.digi.com/resources/library/data-sheets/ds_xbeemultipointmodules Xbee Datasheet]
*[http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems Embedded Drivers & Real Time Operating Systems Lessons]

F21: Treasure Dive

2021-12-18T11:51:04Z

Proj user1:

== '''Treasure Dive''' ==
 
[[File:mermaid.png|thumb|500px|right|Game Screen Capture]]
[[File:Cmpe-244-controller-pic.png|thumb|600px|right|Game Controller]]
 

== Abstract ==
Treasure Dive is a modern one player game inspired by the popular arcade game Breakout, which was released by Atari in 1976. The ancients rambled about an unconfirmed tale of a shipwreck that saw no survivors. As an adventurous and skilled free diver, you are seeking treasure buried deep in an ocean abyss. You dove down all the way to the seafloor and found the buried treasure but need to make it back to the surface to see another day! The player uses a wireless controller to control an on-screen paddle to break bricks on the top of the screen. When enough bricks are broken, a passage is revealed, allowing you to get closer to the surface.

== Objectives & Introduction ==
The objective of this project is to interface an LPC4088 to a VGA monitor by leveraging the LCD controller and use an accelerometer as the sensor for the player to control the paddle.
Two Zigbees RF modules will be integrated for wireless communication between the controller and LPC4088 board via UART protocol.
Digital audio output will be enabled by developing an I2S driver and porting an audio tracker library.
The General Purpose Direct Memory Access (GPDMA) controller memory-to-memory function will be utilized to decrease memory write times when updating the video buffer.
The DMA controller's memory-to-peripheral function will help reduce the frequent I2S interrupts when loading digital audio contents.
An SD card peripheral driver to facilitate memory exchange between an SD card will also be implemented.

=== Team Members & Responsibilities ===
* Brian Ho
** Wiki project schedule planning
** Interfacing the accelerometer
** Digital to Analog Conversion resistor ladder design used to interface digital LCD controller signal with analog VGA port signal
** PCB planning and acquisition
** Procurement of controller hardware
** Wireless communication via Zigbees RF modules and UART protocol
** Wireless controller logic and integration
** Ball movement and wall/brick collision logic
** Hardware/software development and integration
** Final testing and integration
* [https://www.linkedin.com/in/william-lai-525681139 Billy Lai]
** Wiki project schedule planning
** Increased CPU clock rate from 96 MHz to 120 MHz
** Initial memory write timing and measurements to estimate memory write capacity
** DMA driver memory-to-memory implementation for transferring graphics data to video buffer
** I2S with DMA memory-to-peripheral implementation to reduce memory-write interrupts when I2S Tx FIFO level is low
** Paddle movement and paddle collision logic, testing and integration
** Software development and integration
** Final testing and integration
* Jasdip Sekhon
** Wiki project schedule planning
** PCB design and planning
** File loading method algorithm for storing loaded files in memory
** Audio track and audio effects selection for gameplay audio
** Palette manipulation for gameplay
** Software development and integration
** Final testing and integration
* Isaac Wahhab
** Wiki project schedule planning
** DAC resistor ladder design and integration
** Hardware procurement (LPC4088 board, Zigbees, VGA monitor, etc)
** VGA driving through LCD controller
** Audio tracker library port
** Digital audio output using I2S
** Palette scrolling feature implementation
** Graphics rendering
** Vertical scrolling implementation for the video buffer
** Sprite testing and implementation for the ball
** SD card peripheral driver for reading assets from SD card
** Art assets acquisition
** Hardware/software development and integration
** Gameplay logic design
** Gameplay audio and level design
** Final testing and integration

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
*09/21/2021
|
*09/27/2021
|
* Read previous projects, gather information and discuss among the group members.
* Initial ordering of parts (LPC 4088 board, 64x64 LED screen)
|
* Completed
* Completed
|-
! scope="row"| 2
|
*09/28/2021
|
*10/04/2021
|
* Submit project proposals
|
* Completed
|-
! scope="row"| 3
|
*10/05/2021
|
*10/11/2021
|
* Test if board can drive VGA
* Test varying clock rates
* Hardware cursor on VGA
* Rough display driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 4
|
* 10/12/2021
|
* 10/18/2021
|
* DAC resistor ladder
* RGB channels
* Palette shifting
* Measure memory buffer writing timing
* Wiki schedule planning
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
*10/19/2021
|
*10/25/2021
|
* Calculate upper-bound for VGA refresh timings
* Create GitLab repository for project
* Joystick functionality
* SD card driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
*10/26/2021
|
*11/01/2021
|
* Audio peripheral
* Wireless communication using Zigbee module
* PCB planning
* Accelerometer driver
* Continue SD card driver
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/02/2021

|
* 11/08/2021
|
* Continue audio peripheral
* Continue wireless communication using Zigbee module
* Continue accelerometer driver
* PCB finalizing and ordering
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 11/09/2021
|
* 11/15/2021
|
* Convert PNG data to graphics format
* Game background functionality
* Sprite functionality
* User interface
* Integrate components
* Initial testing
|
* Completed
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 11/16/2021
|
* 11/22/2021
|
* Continue everything from last week
* Game physics (wall collision and AI movement)
* Measurements for wooden case
* Art assets (audio and visual)
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 10
|
*11/23/2021
|
*11/29/2021
|

* Fine-tune user experience
* Make case
|
* Completed
* Completed
|-
! scope="row"| 11
|
* 11/30/2021
|
* 12/06/2021
|
* Continue fine-tune user experience
* Integrate hardware
|
* Completed
* Completed
|-
! scope="row"| 12
|
* 12/07/2021
|
* 12/16/2021
|
* Final testing
* Final demo
|
* Completed
* Completed
|-
|}

== Bill Of Materials ==
{| class="wikitable" style="margin-left: 0px; margin-right: auto;"
|-
! scope="col"| Item#
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| [https://www.embeddedartists.com/products/lpc4088-developers-kit LPC4088-32 Developer's Kit]
| 1
| $300.00
|-
! scope="row"| 2
| Microcontroller Board
| [http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems/page/sj2-board SJ2 Board]
| 1
| $50.00
|-
! scope="row"| 3
| Bluetooth Module
| [https://hub.digi.com/support/products/digi-xbee/digi-xbee-s1 Digi XBee-S1]
| 2
| N/A
|-
! scope="row"| 4
| DAC PCB
| [https://jlcpcb.com JLCPCB (Set of 5)]
| 1
| $15.00
|-
! scope="row"| 4
| Monitor
| Asus TFT Monitor
| 1
| N/A
|-
! scope="row"| 5
| Mini Breadboard
| [https://www.amazon.com/EL-CK-002-Electronic-Breadboard-Capacitor-Potentiometer/dp/B01ERP6WL4/ref=pd_sbs_147_2/132-6936347-2117569?_encoding=UTF8&pd_rd_i=B01ERP6WL4&pd_rd_r=6a783095-ec87-42ae-9f01-a5ebc9a44607&pd_rd_w=V2aU6&pd_rd_wg=11h1i&pf_rd_p=d66372fe-68a6-48a3-90ec-41d7f64212be&pf_rd_r=368WNKXFBBQY2FJQ4119&psc=1&refRID=368WNKXFBBQY2FJQ4119 ELEGOO Mini Breadboard]
| 1
| $13.99
|-
! scope="row"| 6
| Arcade buttons
| [https://www.amazon.com/EG-STARTS-Illuminated-Buttons-Raspberry/dp/B01N11BDX9?pd_rd_w=DiSpt&pf_rd_p=5bc65336-a8b0-4888-9e70-4d4ce2f6bd59&pf_rd_r=JCH2VQYXFPHND4XZ4FAM&pd_rd_r=8d6243dd-e1bd-4dae-9b19-be2e00f3085f&pd_rd_wg=K4lt9&pd_rd_i=B01N11BDX9&psc=1&ref_=pd_bap_d_rp_1_i EG STARTS LED Arcade Buttons]
| 2
| $11.99
|-
! scope="row"| 7
| VGA cable
| N/A
| 1
| N/A
|-
! scope="row"| 7
| SD card
| N/A
| 1
| N/A
|-
|}

== Design & Implementation ==

=== Hardware Design ===
We designed our PCB using Autodesk EAGLE based on the tutorial on PCB design during the class lecture. We ordered our PCB to be manufactured from JLCPCB. The PCB consists of an R-2R resistor ladder DAC for the purpose of converting the digital signal from the microcontroller to analog to display graphics on the VGA monitor. Since the VGA input for each color channel is 0-0.7V, the resistor ladder scaled the LPC4088 outputs into proper DAC values for the VGA to take in. Five bits are passed to each of the red, green, and blue channels of the VGA in addition to other signals, such as Horizontal sync (HSYNC) and Vertical sync (VSYNC).
 
[[File:pcbSchematic.png|500px|thumb|left|PCB Schematic]]
[[File:pcbLayout.png|500px|thumb|right|PCB Layout]]
[[File:pcbPic-min.png|500px|thumb|right|Manufactured PCB]]
[[File:pcbManufactured.png|500px|thumb|left|Soldered PCB]]
 

=== Hardware Interface ===

Two Digi XBee devices were used to communicate between the SJ2-Board controller and the LPC4088 board to control the paddle on the game. The XBee devices were programmed using the XTCU programming application provided by Digi International. The SJ2-Board used one XBee as a transmitted and the LPC4088 used another as a receiver. The two modules communicated using UART frames and a single byte represented any change of movement or button presses sent by the controller.

{| class="wikitable"
|-
! scope="col"| XCTU
! scope="col"| Transmitter
! scope="col"| Receiver
|-
! scope="row"| CH
| C
| C
|-
! scope="row"| ID
| 6969
| 6969
|-
! scope="row"| CE
| Coordinator
| Endpoint
|-
! scope="row"| Baud Rate
| 9600
| 9600
|-
|}
[[File:sj2Board.png|380px|thumb|center|SJ2 Board]]
[[File:CMPE146_F16_MW_XbeeModule.jpg|200px|thumb|center|XBee Device]]
 

Two LED buttons were used to interface with the controller as inputs. They were both set up using GPIO.
[[File:VGA LED Button.jpg|200px|thumb|center|LED Button]]
 
[[File:VGA_Controller_Block_Diagram.PNG|380px|thumb|center|Controller Block Diagram]]
 

==== I2S Peripheral and Stereo Audio Decoder ====

The LPC4088 is equipped with an I2S stereo encoder. This module provides a hardware-enabled solution for sending I2S-encoded audio data packets to stereo decoders, which produce the corresponding audio waveform. For this project, since 16-bit stereo audio playback was chosen, data packets transmitted on this interface were sent in the following format:

[[File:I2S_audio_data_word.png]]

Here, each word of data encodes a single packet of audio in two 16-bit sections: one signal level for the left channel, and another for the right. These 4 bytes are sent at a rate of 44100 Hz, resulting in a total data transmission of 176400 bytes a second. Rather than needing to be software-defined, this transmission is automatically handled by hardware using a 8-word deep FIFO queue that manages streaming out the audio data at a fixed rate, saving a significant amount of CPU cycles.

This encoded audio signal was sent to a dedicated external [https://www.nxp.com/docs/en/data-sheet/UDA1380.pdf stereo decoder], which output to a 3.5 mm audio jack connected to speakers in the VGA monitor.

==== LCD Peripheral ====

The LCD peripheral was configured for 16-bit color palette (5 bits of color data for each color level -- red, green, and blue -- and an unused intensity bit), at a resolution of 800 by 600 pixels, and with an active color palette size of 256 colors. Since the logic for driving an LCD display has the same concepts (horizontal blanking, vertical blanking, etc) and timing information for any given resolution was the same, we were able to directly drive a VGA display with the same signals and timing data (after the digital signal had been converted to analog).

[[File:LCD_%2B_VGA_Video_signal.png]]

The timing register configurations needed to achieve the requested video display was derived from a dedicated website for [http://tinyvga.com/vga-timing/800x600@60Hz VGA signal info].

==== External Memory ====

The game's visual assets and tracker audio file (both after being loaded from an SD card) and the game's video buffer required more memory than what is present natively in the microcontroller. To accommodate this, an external 32MiB memory module was used. This module was interfaced with using the external memory controller peripheral present on the board.

=== Software Design ===
==== Controller ====

There is an internal accelerometer on the SJ2 board that is utilized as the movement controls for the paddle movement. The y-axis value of the magnitude is polled 17 times in 17 milliseconds and is divided by 17 to get an average reading per that time frame. The maximum tilt value that can be read by the controller is set to 700 out of the total 1000 (highest possible value). The current tilt value is divided by 31 to get a total of 32 possible readings to represent 5 bits of magnitude.

The controller sends one byte of data to the console whenever there is any change in magnitude on the accelerometer or button changes. The byte contains all the information needed to control the paddle on the screen. The first two bits on the byte represent 0 if there is no button input and 1 if there is a button input. The third bit represents whether the controller is tilted left or right with 0 being left and 1 being right. The last five bits represent the controller accelerometer's magnitude.

[[File:VGA_UART_Frame.PNG]]

==== Audio ====

Sound design was provided to the project (in the form of music and sound effects) through the use of a [https://https://en.wikipedia.org/wiki/Music_tracker tracker audio library]. Source code was adopted from the [https://github.com/schismtracker/schismtracker Schism Tracker] library and altered with the following modifications:

- All UI logic was stripped from the source code

- All file handling logic was reworked to use the SD card reading functions, as the project does not support POSIX file calls

- The [https://www.libsdl.org/ Simple DirectMedia Layer] hooks were replaced with a FreeRTOS task that periodically called the waveform generation callback function, which was passed to the I2S peripheral hooked up to a stereo decoder.

=== Implementation ===

==== Ball Logic ====

The ball is controlled by the buttons and the controller accelerometer values. When the button is pressed, the ball is initially sent with a random velocity. The velocity is determined by a substate timer and is initially set as one of 6 possible values.

{| class="wikitable"
|-
! scope="col"| Substate Count
! scope="col"| X-Velocity
! scope="col"| Y-Velocity
|-
! scope="row"| 0
| 4
| -2
|-
! scope="row"| 1
| -4
| -2
|-
! scope="row"| 2
| 2
| -4
|-
! scope="row"| 3
| -2
| -4
|-
! scope="row"| 4
| 1
| -5
|-
! scope="row"| 5
| -1
| -5
|-
|}

The ball's velocity changes whenever it detects a collision with either the side of the screen, the bricks on the top of the level, or the paddle at the bottom. If colliding with the paddle, the ball's velocity will take into account the paddle's velocity and add it to its current velocity. This is done by adding the paddle's x-velocity and adjusting the y-velocity by normalizing the velocity vectors to have a magnitude of 5. This means the ball will always be moving with a magnitude of 5 in any direction after it hits the paddle. If the ball collides with anything other than the paddle, then the velocity of the ball is changed with no increase or decrease of magnitude. If the ball collides on the side of a wall or block, the x-velocity will reflect (go from negative to positive or vice-versa) and the y-velocity will remain constant. If the ball collides on the top or bottom of a block, the y-velocity will reflect and the x-velocity will remain constant. If it appears that the ball collides with the side and top or bottom of something, then the ball will check whichever face it will collide with first and then calculate the other face. Lastly, when the ball collides with the bottom of the screen, then the player experiences a "death" and the ball will reset on the paddle. If the ball collides with the top of the screen, the screen will scroll up for the user to play on the next level.

Whenever the ball collides with a brick, the brick will disappear. To accomplish this, there is a 2-D array of unsigned integers that contains the values of the bricks. If the integer is 0, then there is no block. If the integer is any value other than 0, then the block is filled and has collision logic. If the ball approaches a region that might have a block, then there will be a check within that 2-D to see if the block is filled or empty. If the block is filled, there will be the collision logic that changes the vector of the ball and sets the integer value in the 2-D array to 0 to represent that the block has been broken.

==== Paddle Movement ====
The paddle movement is controlled by the accelerometer data being wirelessly transmitted from the SJ-Two board to the LPC4088 board. The accelerometer data will map to 63 possible indexes for the paddle location. Originally, the paddle would jump directly into the corresponding index location depending on the accelerometer input. We later iterated on this process by implementing an algorithm to smoothly transition into these indexes. This was done by calculating a paddle velocity using the "current desired paddle position" and the previous paddle position. We adjusted the parameters of the algorithm iteratively to produce the smooth paddle movement.

==== Paddle Collision Logic ====
The position of the ball is calculated by adding delta_x and delta_y to the ball's current x and y position. For each frame, if the ball's next Y position crosses the paddle's Y position, we would check for a paddle collision. To do this, we used the delta_x and delta_y to get the ball's movement as a slope. We then substituted the slope, the ball's current position, and 1 pixel above the paddle's Y position into a point-slope equation (y2 - y1) = m(x2 - x1) to get the paddle's X position at the time of the ball reaching the pixel above paddle's Y position. After finding the location where the ball is expected to collide with the paddle, we then calculated the ball's collision magnitude and the ball's next location's magnitude. Using a ratio of these two magnitudes, we have a frame time fraction of when the ball possibly collided with the paddle. Then, by using the paddle's current position and paddle velocity, we are able to find where the paddle previously was, and measure how far the paddle had moved during that time frame. By multiplying the time frame from the ball's collision magnitude to the paddle's change in X position, we are able to find whether the paddle was under the ball at the time of the ball reaching the pixel above the paddle's Y position.

== Testing & Technical Challenges ==

=== VGA Interfacing ===
A DAC resistor ladder was designed and a corresponding custom PCB was acquired to interface the 3.3V digital LCD controller signal with the analog VGA input signal. The VGA cable takes between 0V (completely dark) and 0.7V (maximum brightness) for color pin inputs. The DAC resistor ladder was designed and implemented on a breadboard to verify the design. Validating the design on a breadboard gave us a clear vision during PCB design and planning.

=== VGA Write Timings ===
There were initial tests that were required to be done to see if using a VGA monitor as an output was a viable option for this project. We had to test memory write timings to see if they were fast enough to match the refresh rate of the monitor. Those tests would decide how much of the screen buffer we could write to during the blanking periods during the VGA refresh. This was a good learning experience for testing hardware capabilities and should be done whenever introducing any new aspects to a project.

=== Ball Collision Logic ===
The initial version of the ball collision used pixel values to see if the ball was entering the brick. The problem was that this logic did not account for the case where the ball enters the brick diagonally and the logic for that version of the collision had to be scrapped. The later iteration of the logic was changed to checking whether the ball was potentially crossing into an area that had a brick. This was done by separating the checks into a grid that represented the locations of the blocks. In this case, there was an initial check to see if the ball was crossing a boundary where there might be a new brick. If the ball is crossing the x-boundary and not the y-boundary, then there would be a collision check to see if it collided with a block. If it is crossing the y-boundary and not the x-boundary, once again then there would be a collision check to see if it collided with a block. If it is crossing both the x-boundary and y-boundary, then there would be a check to see which boundary the ball would cross first. This new logic allowed for us to account for the case where the block enters the block diagonally. This taught us that sometimes it is beneficial to scrap a previous iteration and use a new approach to solving a problem because it may lead to better results.

=== Paddle Movement ===
One issue we discovered was the paddle would slow down as it approached the walls. This was due to the difference in "current desired paddle position" and the previous paddle position becoming smaller as the walls were approached, leading to a smaller paddle velocity. To account for this, whenever the accelerometer sent data that mapped to the minimum or maximum paddle index, we would increase the distance of the "current desired paddle position" so the paddle velocity would not slow the paddle down as it approached the walls.

=== Paddle Collision Logic ===
During final integration and testing, one of the biggest issues we encountered was a bug where the ball would not detect a collision with the paddle. The ball would phase right through the paddle instead of bouncing off of it. One of the great ideas that we thought of was to change the color of the ball during each check to see if the checks would go through. This debug check allowed us to verify that section of code that checked for paddle collisions was being skipped even though the ball's next Y position had crossed the paddle's Y position. By having this debugging method, we were able to root cause the issue.

== Conclusion ==
Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

=== Project Video ===
[https://www.youtube.com/watch?v=ipngAGlN86c SJSU CMPE 244 FALL 2021: Embedded Software Project Gameplay]

=== Project Source Code ===
* [https://gitlab.com/fayek.wahhab/sjtwo-c/-/tree/feature/i2s_dma2 Treasure Dive Baseline for Demo Source Code]
* [https://gitlab.com/hobriandh/sjtwo-c/-/tree/final_controller Controller for Demo Source Code]

== References ==
=== Acknowledgement ===
We would like to acknowledge and thank our instructor Preetpal Kang for sharing his knowledge in embedded software development using FreeRTOS.

=== Appendix ===
*[https://gitlab.com/sjtwo-c-dev/sjtwo-c/-/blob/master/LPC408X_7X%20User%20Manual.pdf LPC408x/407x Datasheet]
*[https://www.freertos.org/Documentation/RTOS_book.html FreeRTOS Documentation]
*[http://elm-chan.org/fsw/ff/00index_e.html FatFs - Generic FAT Filesystem Module]
*[https://www.digi.com/resources/library/data-sheets/ds_xbeemultipointmodules Xbee Datasheet]
*[http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems Embedded Drivers & Real Time Operating Systems Lessons]

F21: Treasure Dive

2021-12-18T11:50:31Z

Proj user1:

[[File:mermaid.png|thumb|500px|right|Game Screen Capture]]
[[File:Cmpe-244-controller-pic.png|thumb|600px|right|Game Controller]]
 

== '''Treasure Dive''' ==

== Abstract ==
Treasure Dive is a modern one player game inspired by the popular arcade game Breakout, which was released by Atari in 1976. The ancients rambled about an unconfirmed tale of a shipwreck that saw no survivors. As an adventurous and skilled free diver, you are seeking treasure buried deep in an ocean abyss. You dove down all the way to the seafloor and found the buried treasure but need to make it back to the surface to see another day! The player uses a wireless controller to control an on-screen paddle to break bricks on the top of the screen. When enough bricks are broken, a passage is revealed, allowing you to get closer to the surface.

== Objectives & Introduction ==
The objective of this project is to interface an LPC4088 to a VGA monitor by leveraging the LCD controller and use an accelerometer as the sensor for the player to control the paddle.
Two Zigbees RF modules will be integrated for wireless communication between the controller and LPC4088 board via UART protocol.
Digital audio output will be enabled by developing an I2S driver and porting an audio tracker library.
The General Purpose Direct Memory Access (GPDMA) controller memory-to-memory function will be utilized to decrease memory write times when updating the video buffer.
The DMA controller's memory-to-peripheral function will help reduce the frequent I2S interrupts when loading digital audio contents.
An SD card peripheral driver to facilitate memory exchange between an SD card will also be implemented.

=== Team Members & Responsibilities ===
* Brian Ho
** Wiki project schedule planning
** Interfacing the accelerometer
** Digital to Analog Conversion resistor ladder design used to interface digital LCD controller signal with analog VGA port signal
** PCB planning and acquisition
** Procurement of controller hardware
** Wireless communication via Zigbees RF modules and UART protocol
** Wireless controller logic and integration
** Ball movement and wall/brick collision logic
** Hardware/software development and integration
** Final testing and integration
* [https://www.linkedin.com/in/william-lai-525681139 Billy Lai]
** Wiki project schedule planning
** Increased CPU clock rate from 96 MHz to 120 MHz
** Initial memory write timing and measurements to estimate memory write capacity
** DMA driver memory-to-memory implementation for transferring graphics data to video buffer
** I2S with DMA memory-to-peripheral implementation to reduce memory-write interrupts when I2S Tx FIFO level is low
** Paddle movement and paddle collision logic, testing and integration
** Software development and integration
** Final testing and integration
* Jasdip Sekhon
** Wiki project schedule planning
** PCB design and planning
** File loading method algorithm for storing loaded files in memory
** Audio track and audio effects selection for gameplay audio
** Palette manipulation for gameplay
** Software development and integration
** Final testing and integration
* Isaac Wahhab
** Wiki project schedule planning
** DAC resistor ladder design and integration
** Hardware procurement (LPC4088 board, Zigbees, VGA monitor, etc)
** VGA driving through LCD controller
** Audio tracker library port
** Digital audio output using I2S
** Palette scrolling feature implementation
** Graphics rendering
** Vertical scrolling implementation for the video buffer
** Sprite testing and implementation for the ball
** SD card peripheral driver for reading assets from SD card
** Art assets acquisition
** Hardware/software development and integration
** Gameplay logic design
** Gameplay audio and level design
** Final testing and integration

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
*09/21/2021
|
*09/27/2021
|
* Read previous projects, gather information and discuss among the group members.
* Initial ordering of parts (LPC 4088 board, 64x64 LED screen)
|
* Completed
* Completed
|-
! scope="row"| 2
|
*09/28/2021
|
*10/04/2021
|
* Submit project proposals
|
* Completed
|-
! scope="row"| 3
|
*10/05/2021
|
*10/11/2021
|
* Test if board can drive VGA
* Test varying clock rates
* Hardware cursor on VGA
* Rough display driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 4
|
* 10/12/2021
|
* 10/18/2021
|
* DAC resistor ladder
* RGB channels
* Palette shifting
* Measure memory buffer writing timing
* Wiki schedule planning
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
*10/19/2021
|
*10/25/2021
|
* Calculate upper-bound for VGA refresh timings
* Create GitLab repository for project
* Joystick functionality
* SD card driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
*10/26/2021
|
*11/01/2021
|
* Audio peripheral
* Wireless communication using Zigbee module
* PCB planning
* Accelerometer driver
* Continue SD card driver
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/02/2021

|
* 11/08/2021
|
* Continue audio peripheral
* Continue wireless communication using Zigbee module
* Continue accelerometer driver
* PCB finalizing and ordering
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 11/09/2021
|
* 11/15/2021
|
* Convert PNG data to graphics format
* Game background functionality
* Sprite functionality
* User interface
* Integrate components
* Initial testing
|
* Completed
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 11/16/2021
|
* 11/22/2021
|
* Continue everything from last week
* Game physics (wall collision and AI movement)
* Measurements for wooden case
* Art assets (audio and visual)
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 10
|
*11/23/2021
|
*11/29/2021
|

* Fine-tune user experience
* Make case
|
* Completed
* Completed
|-
! scope="row"| 11
|
* 11/30/2021
|
* 12/06/2021
|
* Continue fine-tune user experience
* Integrate hardware
|
* Completed
* Completed
|-
! scope="row"| 12
|
* 12/07/2021
|
* 12/16/2021
|
* Final testing
* Final demo
|
* Completed
* Completed
|-
|}

== Bill Of Materials ==
{| class="wikitable" style="margin-left: 0px; margin-right: auto;"
|-
! scope="col"| Item#
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| [https://www.embeddedartists.com/products/lpc4088-developers-kit LPC4088-32 Developer's Kit]
| 1
| $300.00
|-
! scope="row"| 2
| Microcontroller Board
| [http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems/page/sj2-board SJ2 Board]
| 1
| $50.00
|-
! scope="row"| 3
| Bluetooth Module
| [https://hub.digi.com/support/products/digi-xbee/digi-xbee-s1 Digi XBee-S1]
| 2
| N/A
|-
! scope="row"| 4
| DAC PCB
| [https://jlcpcb.com JLCPCB (Set of 5)]
| 1
| $15.00
|-
! scope="row"| 4
| Monitor
| Asus TFT Monitor
| 1
| N/A
|-
! scope="row"| 5
| Mini Breadboard
| [https://www.amazon.com/EL-CK-002-Electronic-Breadboard-Capacitor-Potentiometer/dp/B01ERP6WL4/ref=pd_sbs_147_2/132-6936347-2117569?_encoding=UTF8&pd_rd_i=B01ERP6WL4&pd_rd_r=6a783095-ec87-42ae-9f01-a5ebc9a44607&pd_rd_w=V2aU6&pd_rd_wg=11h1i&pf_rd_p=d66372fe-68a6-48a3-90ec-41d7f64212be&pf_rd_r=368WNKXFBBQY2FJQ4119&psc=1&refRID=368WNKXFBBQY2FJQ4119 ELEGOO Mini Breadboard]
| 1
| $13.99
|-
! scope="row"| 6
| Arcade buttons
| [https://www.amazon.com/EG-STARTS-Illuminated-Buttons-Raspberry/dp/B01N11BDX9?pd_rd_w=DiSpt&pf_rd_p=5bc65336-a8b0-4888-9e70-4d4ce2f6bd59&pf_rd_r=JCH2VQYXFPHND4XZ4FAM&pd_rd_r=8d6243dd-e1bd-4dae-9b19-be2e00f3085f&pd_rd_wg=K4lt9&pd_rd_i=B01N11BDX9&psc=1&ref_=pd_bap_d_rp_1_i EG STARTS LED Arcade Buttons]
| 2
| $11.99
|-
! scope="row"| 7
| VGA cable
| N/A
| 1
| N/A
|-
! scope="row"| 7
| SD card
| N/A
| 1
| N/A
|-
|}

== Design & Implementation ==

=== Hardware Design ===
We designed our PCB using Autodesk EAGLE based on the tutorial on PCB design during the class lecture. We ordered our PCB to be manufactured from JLCPCB. The PCB consists of an R-2R resistor ladder DAC for the purpose of converting the digital signal from the microcontroller to analog to display graphics on the VGA monitor. Since the VGA input for each color channel is 0-0.7V, the resistor ladder scaled the LPC4088 outputs into proper DAC values for the VGA to take in. Five bits are passed to each of the red, green, and blue channels of the VGA in addition to other signals, such as Horizontal sync (HSYNC) and Vertical sync (VSYNC).
 
[[File:pcbSchematic.png|500px|thumb|left|PCB Schematic]]
[[File:pcbLayout.png|500px|thumb|right|PCB Layout]]
[[File:pcbPic-min.png|500px|thumb|right|Manufactured PCB]]
[[File:pcbManufactured.png|500px|thumb|left|Soldered PCB]]
 

=== Hardware Interface ===

Two Digi XBee devices were used to communicate between the SJ2-Board controller and the LPC4088 board to control the paddle on the game. The XBee devices were programmed using the XTCU programming application provided by Digi International. The SJ2-Board used one XBee as a transmitted and the LPC4088 used another as a receiver. The two modules communicated using UART frames and a single byte represented any change of movement or button presses sent by the controller.

{| class="wikitable"
|-
! scope="col"| XCTU
! scope="col"| Transmitter
! scope="col"| Receiver
|-
! scope="row"| CH
| C
| C
|-
! scope="row"| ID
| 6969
| 6969
|-
! scope="row"| CE
| Coordinator
| Endpoint
|-
! scope="row"| Baud Rate
| 9600
| 9600
|-
|}
[[File:sj2Board.png|380px|thumb|center|SJ2 Board]]
[[File:CMPE146_F16_MW_XbeeModule.jpg|200px|thumb|center|XBee Device]]
 

Two LED buttons were used to interface with the controller as inputs. They were both set up using GPIO.
[[File:VGA LED Button.jpg|200px|thumb|center|LED Button]]
 
[[File:VGA_Controller_Block_Diagram.PNG|380px|thumb|center|Controller Block Diagram]]
 

==== I2S Peripheral and Stereo Audio Decoder ====

The LPC4088 is equipped with an I2S stereo encoder. This module provides a hardware-enabled solution for sending I2S-encoded audio data packets to stereo decoders, which produce the corresponding audio waveform. For this project, since 16-bit stereo audio playback was chosen, data packets transmitted on this interface were sent in the following format:

[[File:I2S_audio_data_word.png]]

Here, each word of data encodes a single packet of audio in two 16-bit sections: one signal level for the left channel, and another for the right. These 4 bytes are sent at a rate of 44100 Hz, resulting in a total data transmission of 176400 bytes a second. Rather than needing to be software-defined, this transmission is automatically handled by hardware using a 8-word deep FIFO queue that manages streaming out the audio data at a fixed rate, saving a significant amount of CPU cycles.

This encoded audio signal was sent to a dedicated external [https://www.nxp.com/docs/en/data-sheet/UDA1380.pdf stereo decoder], which output to a 3.5 mm audio jack connected to speakers in the VGA monitor.

==== LCD Peripheral ====

The LCD peripheral was configured for 16-bit color palette (5 bits of color data for each color level -- red, green, and blue -- and an unused intensity bit), at a resolution of 800 by 600 pixels, and with an active color palette size of 256 colors. Since the logic for driving an LCD display has the same concepts (horizontal blanking, vertical blanking, etc) and timing information for any given resolution was the same, we were able to directly drive a VGA display with the same signals and timing data (after the digital signal had been converted to analog).

[[File:LCD_%2B_VGA_Video_signal.png]]

The timing register configurations needed to achieve the requested video display was derived from a dedicated website for [http://tinyvga.com/vga-timing/800x600@60Hz VGA signal info].

==== External Memory ====

The game's visual assets and tracker audio file (both after being loaded from an SD card) and the game's video buffer required more memory than what is present natively in the microcontroller. To accommodate this, an external 32MiB memory module was used. This module was interfaced with using the external memory controller peripheral present on the board.

=== Software Design ===
==== Controller ====

There is an internal accelerometer on the SJ2 board that is utilized as the movement controls for the paddle movement. The y-axis value of the magnitude is polled 17 times in 17 milliseconds and is divided by 17 to get an average reading per that time frame. The maximum tilt value that can be read by the controller is set to 700 out of the total 1000 (highest possible value). The current tilt value is divided by 31 to get a total of 32 possible readings to represent 5 bits of magnitude.

The controller sends one byte of data to the console whenever there is any change in magnitude on the accelerometer or button changes. The byte contains all the information needed to control the paddle on the screen. The first two bits on the byte represent 0 if there is no button input and 1 if there is a button input. The third bit represents whether the controller is tilted left or right with 0 being left and 1 being right. The last five bits represent the controller accelerometer's magnitude.

[[File:VGA_UART_Frame.PNG]]

==== Audio ====

Sound design was provided to the project (in the form of music and sound effects) through the use of a [https://https://en.wikipedia.org/wiki/Music_tracker tracker audio library]. Source code was adopted from the [https://github.com/schismtracker/schismtracker Schism Tracker] library and altered with the following modifications:

- All UI logic was stripped from the source code

- All file handling logic was reworked to use the SD card reading functions, as the project does not support POSIX file calls

- The [https://www.libsdl.org/ Simple DirectMedia Layer] hooks were replaced with a FreeRTOS task that periodically called the waveform generation callback function, which was passed to the I2S peripheral hooked up to a stereo decoder.

=== Implementation ===

==== Ball Logic ====

The ball is controlled by the buttons and the controller accelerometer values. When the button is pressed, the ball is initially sent with a random velocity. The velocity is determined by a substate timer and is initially set as one of 6 possible values.

{| class="wikitable"
|-
! scope="col"| Substate Count
! scope="col"| X-Velocity
! scope="col"| Y-Velocity
|-
! scope="row"| 0
| 4
| -2
|-
! scope="row"| 1
| -4
| -2
|-
! scope="row"| 2
| 2
| -4
|-
! scope="row"| 3
| -2
| -4
|-
! scope="row"| 4
| 1
| -5
|-
! scope="row"| 5
| -1
| -5
|-
|}

The ball's velocity changes whenever it detects a collision with either the side of the screen, the bricks on the top of the level, or the paddle at the bottom. If colliding with the paddle, the ball's velocity will take into account the paddle's velocity and add it to its current velocity. This is done by adding the paddle's x-velocity and adjusting the y-velocity by normalizing the velocity vectors to have a magnitude of 5. This means the ball will always be moving with a magnitude of 5 in any direction after it hits the paddle. If the ball collides with anything other than the paddle, then the velocity of the ball is changed with no increase or decrease of magnitude. If the ball collides on the side of a wall or block, the x-velocity will reflect (go from negative to positive or vice-versa) and the y-velocity will remain constant. If the ball collides on the top or bottom of a block, the y-velocity will reflect and the x-velocity will remain constant. If it appears that the ball collides with the side and top or bottom of something, then the ball will check whichever face it will collide with first and then calculate the other face. Lastly, when the ball collides with the bottom of the screen, then the player experiences a "death" and the ball will reset on the paddle. If the ball collides with the top of the screen, the screen will scroll up for the user to play on the next level.

Whenever the ball collides with a brick, the brick will disappear. To accomplish this, there is a 2-D array of unsigned integers that contains the values of the bricks. If the integer is 0, then there is no block. If the integer is any value other than 0, then the block is filled and has collision logic. If the ball approaches a region that might have a block, then there will be a check within that 2-D to see if the block is filled or empty. If the block is filled, there will be the collision logic that changes the vector of the ball and sets the integer value in the 2-D array to 0 to represent that the block has been broken.

==== Paddle Movement ====
The paddle movement is controlled by the accelerometer data being wirelessly transmitted from the SJ-Two board to the LPC4088 board. The accelerometer data will map to 63 possible indexes for the paddle location. Originally, the paddle would jump directly into the corresponding index location depending on the accelerometer input. We later iterated on this process by implementing an algorithm to smoothly transition into these indexes. This was done by calculating a paddle velocity using the "current desired paddle position" and the previous paddle position. We adjusted the parameters of the algorithm iteratively to produce the smooth paddle movement.

==== Paddle Collision Logic ====
The position of the ball is calculated by adding delta_x and delta_y to the ball's current x and y position. For each frame, if the ball's next Y position crosses the paddle's Y position, we would check for a paddle collision. To do this, we used the delta_x and delta_y to get the ball's movement as a slope. We then substituted the slope, the ball's current position, and 1 pixel above the paddle's Y position into a point-slope equation (y2 - y1) = m(x2 - x1) to get the paddle's X position at the time of the ball reaching the pixel above paddle's Y position. After finding the location where the ball is expected to collide with the paddle, we then calculated the ball's collision magnitude and the ball's next location's magnitude. Using a ratio of these two magnitudes, we have a frame time fraction of when the ball possibly collided with the paddle. Then, by using the paddle's current position and paddle velocity, we are able to find where the paddle previously was, and measure how far the paddle had moved during that time frame. By multiplying the time frame from the ball's collision magnitude to the paddle's change in X position, we are able to find whether the paddle was under the ball at the time of the ball reaching the pixel above the paddle's Y position.

== Testing & Technical Challenges ==

=== VGA Interfacing ===
A DAC resistor ladder was designed and a corresponding custom PCB was acquired to interface the 3.3V digital LCD controller signal with the analog VGA input signal. The VGA cable takes between 0V (completely dark) and 0.7V (maximum brightness) for color pin inputs. The DAC resistor ladder was designed and implemented on a breadboard to verify the design. Validating the design on a breadboard gave us a clear vision during PCB design and planning.

=== VGA Write Timings ===
There were initial tests that were required to be done to see if using a VGA monitor as an output was a viable option for this project. We had to test memory write timings to see if they were fast enough to match the refresh rate of the monitor. Those tests would decide how much of the screen buffer we could write to during the blanking periods during the VGA refresh. This was a good learning experience for testing hardware capabilities and should be done whenever introducing any new aspects to a project.

=== Ball Collision Logic ===
The initial version of the ball collision used pixel values to see if the ball was entering the brick. The problem was that this logic did not account for the case where the ball enters the brick diagonally and the logic for that version of the collision had to be scrapped. The later iteration of the logic was changed to checking whether the ball was potentially crossing into an area that had a brick. This was done by separating the checks into a grid that represented the locations of the blocks. In this case, there was an initial check to see if the ball was crossing a boundary where there might be a new brick. If the ball is crossing the x-boundary and not the y-boundary, then there would be a collision check to see if it collided with a block. If it is crossing the y-boundary and not the x-boundary, once again then there would be a collision check to see if it collided with a block. If it is crossing both the x-boundary and y-boundary, then there would be a check to see which boundary the ball would cross first. This new logic allowed for us to account for the case where the block enters the block diagonally. This taught us that sometimes it is beneficial to scrap a previous iteration and use a new approach to solving a problem because it may lead to better results.

=== Paddle Movement ===
One issue we discovered was the paddle would slow down as it approached the walls. This was due to the difference in "current desired paddle position" and the previous paddle position becoming smaller as the walls were approached, leading to a smaller paddle velocity. To account for this, whenever the accelerometer sent data that mapped to the minimum or maximum paddle index, we would increase the distance of the "current desired paddle position" so the paddle velocity would not slow the paddle down as it approached the walls.

=== Paddle Collision Logic ===
During final integration and testing, one of the biggest issues we encountered was a bug where the ball would not detect a collision with the paddle. The ball would phase right through the paddle instead of bouncing off of it. One of the great ideas that we thought of was to change the color of the ball during each check to see if the checks would go through. This debug check allowed us to verify that section of code that checked for paddle collisions was being skipped even though the ball's next Y position had crossed the paddle's Y position. By having this debugging method, we were able to root cause the issue.

== Conclusion ==
Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

=== Project Video ===
[https://www.youtube.com/watch?v=ipngAGlN86c SJSU CMPE 244 FALL 2021: Embedded Software Project Gameplay]

=== Project Source Code ===
* [https://gitlab.com/fayek.wahhab/sjtwo-c/-/tree/feature/i2s_dma2 Treasure Dive Baseline for Demo Source Code]
* [https://gitlab.com/hobriandh/sjtwo-c/-/tree/final_controller Controller for Demo Source Code]

== References ==
=== Acknowledgement ===
We would like to acknowledge and thank our instructor Preetpal Kang for sharing his knowledge in embedded software development using FreeRTOS.

=== Appendix ===
*[https://gitlab.com/sjtwo-c-dev/sjtwo-c/-/blob/master/LPC408X_7X%20User%20Manual.pdf LPC408x/407x Datasheet]
*[https://www.freertos.org/Documentation/RTOS_book.html FreeRTOS Documentation]
*[http://elm-chan.org/fsw/ff/00index_e.html FatFs - Generic FAT Filesystem Module]
*[https://www.digi.com/resources/library/data-sheets/ds_xbeemultipointmodules Xbee Datasheet]
*[http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems Embedded Drivers & Real Time Operating Systems Lessons]

F21: Treasure Dive

2021-12-18T11:46:06Z

Proj user1: /* Team Members & Responsibilities */

[[File:mermaid.png|thumb|500px|right|Game Screen Capture]]
[[File:Cmpe-244-controller-pic.png|thumb|600px|right|Game Controller]]
 
=== Grading Criteria ===

* How well is Software & Hardware Design described?
* How well can this report be used to reproduce this project?
* Code Quality
* Overall Report Quality:
** Software Block Diagrams
** Hardware Block Diagrams
**: Schematic Quality
** Quality of technical challenges and solutions adopted.


== '''Treasure Dive''' ==

== Abstract ==
Treasure Dive is a modern one player game inspired by the popular arcade game Breakout, which was released by Atari in 1976. The ancients rambled about an unconfirmed tale of a shipwreck that saw no survivors. As an adventurous and skilled free diver, you are seeking treasure buried deep in an ocean abyss. You dove down all the way to the seafloor and found the buried treasure but need to make it back to the surface to see another day! The player uses a wireless controller to control an on-screen paddle to break bricks on the top of the screen. When enough bricks are broken, a passage is revealed, allowing you to get closer to the surface.

== Objectives & Introduction ==
The objective of this project is to interface an LPC4088 to a VGA monitor by leveraging the LCD controller and use an accelerometer as the sensor for the player to control the paddle.
Two Zigbees RF modules will be integrated for wireless communication between the controller and LPC4088 board via UART protocol.
Digital audio output will be enabled by developing an I2S driver and porting an audio tracker library.
The General Purpose Direct Memory Access (GPDMA) controller memory-to-memory function will be utilized to decrease memory write times when updating the video buffer.
The DMA controller's memory-to-peripheral function will help reduce the frequent I2S interrupts when loading digital audio contents.
An SD card peripheral driver to facilitate memory exchange between an SD card will also be implemented.

=== Team Members & Responsibilities ===
* Brian Ho
** Wiki project schedule planning
** Interfacing the accelerometer
** Digital to Analog Conversion resistor ladder design used to interface digital LCD controller signal with analog VGA port signal
** PCB planning and acquisition
** Procurement of controller hardware
** Wireless communication via Zigbees RF modules and UART protocol
** Wireless controller logic and integration
** Ball movement and wall/brick collision logic
** Hardware/software development and integration
** Final testing and integration
* [https://www.linkedin.com/in/william-lai-525681139 Billy Lai]
** Wiki project schedule planning
** Increased CPU clock rate from 96 MHz to 120 MHz
** Initial memory write timing and measurements to estimate memory write capacity
** DMA driver memory-to-memory implementation for transferring graphics data to video buffer
** I2S with DMA memory-to-peripheral implementation to reduce memory-write interrupts when I2S Tx FIFO level is low
** Paddle movement and paddle collision logic, testing and integration
** Software development and integration
** Final testing and integration
* Jasdip Sekhon
** Wiki project schedule planning
** PCB design and planning
** File loading method algorithm for storing loaded files in memory
** Audio track and audio effects selection for gameplay audio
** Palette manipulation for gameplay
** Software development and integration
** Final testing and integration
* Isaac Wahhab
** Wiki project schedule planning
** DAC resistor ladder design and integration
** Hardware procurement (LPC4088 board, Zigbees, VGA monitor, etc)
** VGA driving through LCD controller
** Audio tracker library port
** Digital audio output using I2S
** Palette scrolling feature implementation
** Graphics rendering
** Vertical scrolling implementation for the video buffer
** Sprite testing and implementation for the ball
** SD card peripheral driver for reading assets from SD card
** Art assets acquisition
** Hardware/software development and integration
** Gameplay logic design
** Gameplay audio and level design
** Final testing and integration

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
*09/21/2021
|
*09/27/2021
|
* Read previous projects, gather information and discuss among the group members.
* Initial ordering of parts (LPC 4088 board, 64x64 LED screen)
|
* Completed
* Completed
|-
! scope="row"| 2
|
*09/28/2021
|
*10/04/2021
|
* Submit project proposals
|
* Completed
|-
! scope="row"| 3
|
*10/05/2021
|
*10/11/2021
|
* Test if board can drive VGA
* Test varying clock rates
* Hardware cursor on VGA
* Rough display driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 4
|
* 10/12/2021
|
* 10/18/2021
|
* DAC resistor ladder
* RGB channels
* Palette shifting
* Measure memory buffer writing timing
* Wiki schedule planning
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
*10/19/2021
|
*10/25/2021
|
* Calculate upper-bound for VGA refresh timings
* Create GitLab repository for project
* Joystick functionality
* SD card driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
*10/26/2021
|
*11/01/2021
|
* Audio peripheral
* Wireless communication using Zigbee module
* PCB planning
* Accelerometer driver
* Continue SD card driver
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/02/2021

|
* 11/08/2021
|
* Continue audio peripheral
* Continue wireless communication using Zigbee module
* Continue accelerometer driver
* PCB finalizing and ordering
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 11/09/2021
|
* 11/15/2021
|
* Convert PNG data to graphics format
* Game background functionality
* Sprite functionality
* User interface
* Integrate components
* Initial testing
|
* Completed
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 11/16/2021
|
* 11/22/2021
|
* Continue everything from last week
* Game physics (wall collision and AI movement)
* Measurements for wooden case
* Art assets (audio and visual)
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 10
|
*11/23/2021
|
*11/29/2021
|

* Fine-tune user experience
* Make case
|
* Completed
* Completed
|-
! scope="row"| 11
|
* 11/30/2021
|
* 12/06/2021
|
* Continue fine-tune user experience
* Integrate hardware
|
* Completed
* Completed
|-
! scope="row"| 12
|
* 12/07/2021
|
* 12/16/2021
|
* Final testing
* Final demo
|
* Completed
* Completed
|-
|}

== Bill Of Materials ==
{| class="wikitable" style="margin-left: 0px; margin-right: auto;"
|-
! scope="col"| Item#
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| [https://www.embeddedartists.com/products/lpc4088-developers-kit LPC4088-32 Developer's Kit]
| 1
| $300.00
|-
! scope="row"| 2
| Microcontroller Board
| [http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems/page/sj2-board SJ2 Board]
| 1
| $50.00
|-
! scope="row"| 3
| Bluetooth Module
| [https://hub.digi.com/support/products/digi-xbee/digi-xbee-s1 Digi XBee-S1]
| 2
| N/A
|-
! scope="row"| 4
| DAC PCB
| [https://jlcpcb.com JLCPCB (Set of 5)]
| 1
| $15.00
|-
! scope="row"| 4
| Monitor
| Asus TFT Monitor
| 1
| N/A
|-
! scope="row"| 5
| Mini Breadboard
| [https://www.amazon.com/EL-CK-002-Electronic-Breadboard-Capacitor-Potentiometer/dp/B01ERP6WL4/ref=pd_sbs_147_2/132-6936347-2117569?_encoding=UTF8&pd_rd_i=B01ERP6WL4&pd_rd_r=6a783095-ec87-42ae-9f01-a5ebc9a44607&pd_rd_w=V2aU6&pd_rd_wg=11h1i&pf_rd_p=d66372fe-68a6-48a3-90ec-41d7f64212be&pf_rd_r=368WNKXFBBQY2FJQ4119&psc=1&refRID=368WNKXFBBQY2FJQ4119 ELEGOO Mini Breadboard]
| 1
| $13.99
|-
! scope="row"| 6
| Arcade buttons
| [https://www.amazon.com/EG-STARTS-Illuminated-Buttons-Raspberry/dp/B01N11BDX9?pd_rd_w=DiSpt&pf_rd_p=5bc65336-a8b0-4888-9e70-4d4ce2f6bd59&pf_rd_r=JCH2VQYXFPHND4XZ4FAM&pd_rd_r=8d6243dd-e1bd-4dae-9b19-be2e00f3085f&pd_rd_wg=K4lt9&pd_rd_i=B01N11BDX9&psc=1&ref_=pd_bap_d_rp_1_i EG STARTS LED Arcade Buttons]
| 2
| $11.99
|-
! scope="row"| 7
| VGA cable
| N/A
| 1
| N/A
|-
! scope="row"| 7
| SD card
| N/A
| 1
| N/A
|-
|}

== Design & Implementation ==

=== Hardware Design ===
We designed our PCB using Autodesk EAGLE based on the tutorial on PCB design during the class lecture. We ordered our PCB to be manufactured from JLCPCB. The PCB consists of an R-2R resistor ladder DAC for the purpose of converting the digital signal from the microcontroller to analog to display graphics on the VGA monitor. Since the VGA input for each color channel is 0-0.7V, the resistor ladder scaled the LPC4088 outputs into proper DAC values for the VGA to take in. Five bits are passed to each of the red, green, and blue channels of the VGA in addition to other signals, such as Horizontal sync (HSYNC) and Vertical sync (VSYNC).
 
[[File:pcbSchematic.png|500px|thumb|left|PCB Schematic]]
[[File:pcbLayout.png|500px|thumb|right|PCB Layout]]
[[File:pcbPic-min.png|500px|thumb|right|Manufactured PCB]]
[[File:pcbManufactured.png|500px|thumb|left|Soldered PCB]]
 

=== Hardware Interface ===

Two Digi XBee devices were used to communicate between the SJ2-Board controller and the LPC4088 board to control the paddle on the game. The XBee devices were programmed using the XTCU programming application provided by Digi International. The SJ2-Board used one XBee as a transmitted and the LPC4088 used another as a receiver. The two modules communicated using UART frames and a single byte represented any change of movement or button presses sent by the controller.

{| class="wikitable"
|-
! scope="col"| XCTU
! scope="col"| Transmitter
! scope="col"| Receiver
|-
! scope="row"| CH
| C
| C
|-
! scope="row"| ID
| 6969
| 6969
|-
! scope="row"| CE
| Coordinator
| Endpoint
|-
! scope="row"| Baud Rate
| 9600
| 9600
|-
|}
[[File:sj2Board.png|380px|thumb|center|SJ2 Board]]
[[File:CMPE146_F16_MW_XbeeModule.jpg|200px|thumb|center|XBee Device]]
 

Two LED buttons were used to interface with the controller as inputs. They were both set up using GPIO.
[[File:VGA LED Button.jpg|200px|thumb|center|LED Button]]
 
[[File:VGA_Controller_Block_Diagram.PNG|380px|thumb|center|Controller Block Diagram]]
 

==== I2S Peripheral and Stereo Audio Decoder ====

The LPC4088 is equipped with an I2S stereo encoder. This module provides a hardware-enabled solution for sending I2S-encoded audio data packets to stereo decoders, which produce the corresponding audio waveform. For this project, since 16-bit stereo audio playback was chosen, data packets transmitted on this interface were sent in the following format:

[[File:I2S_audio_data_word.png]]

Here, each word of data encodes a single packet of audio in two 16-bit sections: one signal level for the left channel, and another for the right. These 4 bytes are sent at a rate of 44100 Hz, resulting in a total data transmission of 176400 bytes a second. Rather than needing to be software-defined, this transmission is automatically handled by hardware using a 8-word deep FIFO queue that manages streaming out the audio data at a fixed rate, saving a significant amount of CPU cycles.

This encoded audio signal was sent to a dedicated external [https://www.nxp.com/docs/en/data-sheet/UDA1380.pdf stereo decoder], which output to a 3.5 mm audio jack connected to speakers in the VGA monitor.

==== LCD Peripheral ====

The LCD peripheral was configured for 16-bit color palette (5 bits of color data for each color level -- red, green, and blue -- and an unused intensity bit), at a resolution of 800 by 600 pixels, and with an active color palette size of 256 colors. Since the logic for driving an LCD display has the same concepts (horizontal blanking, vertical blanking, etc) and timing information for any given resolution was the same, we were able to directly drive a VGA display with the same signals and timing data (after the digital signal had been converted to analog).

[[File:LCD_%2B_VGA_Video_signal.png]]

The timing register configurations needed to achieve the requested video display was derived from a dedicated website for [http://tinyvga.com/vga-timing/800x600@60Hz VGA signal info].

==== External Memory ====

The game's visual assets and tracker audio file (both after being loaded from an SD card) and the game's video buffer required more memory than what is present natively in the microcontroller. To accommodate this, an external 32MiB memory module was used. This module was interfaced with using the external memory controller peripheral present on the board.

=== Software Design ===
==== Controller ====

There is an internal accelerometer on the SJ2 board that is utilized as the movement controls for the paddle movement. The y-axis value of the magnitude is polled 17 times in 17 milliseconds and is divided by 17 to get an average reading per that time frame. The maximum tilt value that can be read by the controller is set to 700 out of the total 1000 (highest possible value). The current tilt value is divided by 31 to get a total of 32 possible readings to represent 5 bits of magnitude.

The controller sends one byte of data to the console whenever there is any change in magnitude on the accelerometer or button changes. The byte contains all the information needed to control the paddle on the screen. The first two bits on the byte represent 0 if there is no button input and 1 if there is a button input. The third bit represents whether the controller is tilted left or right with 0 being left and 1 being right. The last five bits represent the controller accelerometer's magnitude.

[[File:VGA_UART_Frame.PNG]]

==== Audio ====

Sound design was provided to the project (in the form of music and sound effects) through the use of a [https://https://en.wikipedia.org/wiki/Music_tracker tracker audio library]. Source code was adopted from the [https://github.com/schismtracker/schismtracker Schism Tracker] library and altered with the following modifications:

- All UI logic was stripped from the source code

- All file handling logic was reworked to use the SD card reading functions, as the project does not support POSIX file calls

- The [https://www.libsdl.org/ Simple DirectMedia Layer] hooks were replaced with a FreeRTOS task that periodically called the waveform generation callback function, which was passed to the I2S peripheral hooked up to a stereo decoder.

=== Implementation ===

==== Ball Logic ====

The ball is controlled by the buttons and the controller accelerometer values. When the button is pressed, the ball is initially sent with a random velocity. The velocity is determined by a substate timer and is initially set as one of 6 possible values.

{| class="wikitable"
|-
! scope="col"| Substate Count
! scope="col"| X-Velocity
! scope="col"| Y-Velocity
|-
! scope="row"| 0
| 4
| -2
|-
! scope="row"| 1
| -4
| -2
|-
! scope="row"| 2
| 2
| -4
|-
! scope="row"| 3
| -2
| -4
|-
! scope="row"| 4
| 1
| -5
|-
! scope="row"| 5
| -1
| -5
|-
|}

The ball's velocity changes whenever it detects a collision with either the side of the screen, the bricks on the top of the level, or the paddle at the bottom. If colliding with the paddle, the ball's velocity will take into account the paddle's velocity and add it to its current velocity. This is done by adding the paddle's x-velocity and adjusting the y-velocity by normalizing the velocity vectors to have a magnitude of 5. This means the ball will always be moving with a magnitude of 5 in any direction after it hits the paddle. If the ball collides with anything other than the paddle, then the velocity of the ball is changed with no increase or decrease of magnitude. If the ball collides on the side of a wall or block, the x-velocity will reflect (go from negative to positive or vice-versa) and the y-velocity will remain constant. If the ball collides on the top or bottom of a block, the y-velocity will reflect and the x-velocity will remain constant. If it appears that the ball collides with the side and top or bottom of something, then the ball will check whichever face it will collide with first and then calculate the other face. Lastly, when the ball collides with the bottom of the screen, then the player experiences a "death" and the ball will reset on the paddle. If the ball collides with the top of the screen, the screen will scroll up for the user to play on the next level.

Whenever the ball collides with a brick, the brick will disappear. To accomplish this, there is a 2-D array of unsigned integers that contains the values of the bricks. If the integer is 0, then there is no block. If the integer is any value other than 0, then the block is filled and has collision logic. If the ball approaches a region that might have a block, then there will be a check within that 2-D to see if the block is filled or empty. If the block is filled, there will be the collision logic that changes the vector of the ball and sets the integer value in the 2-D array to 0 to represent that the block has been broken.

==== Paddle Movement ====
The paddle movement is controlled by the accelerometer data being wirelessly transmitted from the SJ-Two board to the LPC4088 board. The accelerometer data will map to 63 possible indexes for the paddle location. Originally, the paddle would jump directly into the corresponding index location depending on the accelerometer input. We later iterated on this process by implementing an algorithm to smoothly transition into these indexes. This was done by calculating a paddle velocity using the "current desired paddle position" and the previous paddle position. We adjusted the parameters of the algorithm iteratively to produce the smooth paddle movement.

==== Paddle Collision Logic ====
The position of the ball is calculated by adding delta_x and delta_y to the ball's current x and y position. For each frame, if the ball's next Y position crosses the paddle's Y position, we would check for a paddle collision. To do this, we used the delta_x and delta_y to get the ball's movement as a slope. We then substituted the slope, the ball's current position, and 1 pixel above the paddle's Y position into a point-slope equation (y2 - y1) = m(x2 - x1) to get the paddle's X position at the time of the ball reaching the pixel above paddle's Y position. After finding the location where the ball is expected to collide with the paddle, we then calculated the ball's collision magnitude and the ball's next location's magnitude. Using a ratio of these two magnitudes, we have a frame time fraction of when the ball possibly collided with the paddle. Then, by using the paddle's current position and paddle velocity, we are able to find where the paddle previously was, and measure how far the paddle had moved during that time frame. By multiplying the time frame from the ball's collision magnitude to the paddle's change in X position, we are able to find whether the paddle was under the ball at the time of the ball reaching the pixel above the paddle's Y position.

== Testing & Technical Challenges ==

=== VGA Interfacing ===
A DAC resistor ladder was designed and a corresponding custom PCB was acquired to interface the 3.3V digital LCD controller signal with the analog VGA input signal. The VGA cable takes between 0V (completely dark) and 0.7V (maximum brightness) for color pin inputs. The DAC resistor ladder was designed and implemented on a breadboard to verify the design. Validating the design on a breadboard gave us a clear vision during PCB design and planning.

=== VGA Write Timings ===
There were initial tests that were required to be done to see if using a VGA monitor as an output was a viable option for this project. We had to test memory write timings to see if they were fast enough to match the refresh rate of the monitor. Those tests would decide how much of the screen buffer we could write to during the blanking periods during the VGA refresh. This was a good learning experience for testing hardware capabilities and should be done whenever introducing any new aspects to a project.

=== Ball Collision Logic ===
The initial version of the ball collision used pixel values to see if the ball was entering the brick. The problem was that this logic did not account for the case where the ball enters the brick diagonally and the logic for that version of the collision had to be scrapped. The later iteration of the logic was changed to checking whether the ball was potentially crossing into an area that had a brick. This was done by separating the checks into a grid that represented the locations of the blocks. In this case, there was an initial check to see if the ball was crossing a boundary where there might be a new brick. If the ball is crossing the x-boundary and not the y-boundary, then there would be a collision check to see if it collided with a block. If it is crossing the y-boundary and not the x-boundary, once again then there would be a collision check to see if it collided with a block. If it is crossing both the x-boundary and y-boundary, then there would be a check to see which boundary the ball would cross first. This new logic allowed for us to account for the case where the block enters the block diagonally. This taught us that sometimes it is beneficial to scrap a previous iteration and use a new approach to solving a problem because it may lead to better results.

=== Paddle Movement ===
One issue we discovered was the paddle would slow down as it approached the walls. This was due to the difference in "current desired paddle position" and the previous paddle position becoming smaller as the walls were approached, leading to a smaller paddle velocity. To account for this, whenever the accelerometer sent data that mapped to the minimum or maximum paddle index, we would increase the distance of the "current desired paddle position" so the paddle velocity would not slow the paddle down as it approached the walls.

=== Paddle Collision Logic ===
During final integration and testing, one of the biggest issues we encountered was a bug where the ball would not detect a collision with the paddle. The ball would phase right through the paddle instead of bouncing off of it. One of the great ideas that we thought of was to change the color of the ball during each check to see if the checks would go through. This debug check allowed us to verify that section of code that checked for paddle collisions was being skipped even though the ball's next Y position had crossed the paddle's Y position. By having this debugging method, we were able to root cause the issue.

== Conclusion ==
Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

=== Project Video ===
[https://www.youtube.com/watch?v=ipngAGlN86c SJSU CMPE 244 FALL 2021: Embedded Software Project Gameplay]

=== Project Source Code ===
* [https://gitlab.com/fayek.wahhab/sjtwo-c/-/tree/feature/i2s_dma2 Treasure Dive Baseline for Demo Source Code]
* [https://gitlab.com/hobriandh/sjtwo-c/-/tree/final_controller Controller for Demo Source Code]

== References ==
=== Acknowledgement ===
We would like to acknowledge and thank our instructor Preetpal Kang for sharing his knowledge in embedded software development using FreeRTOS.

=== Appendix ===
*[https://gitlab.com/sjtwo-c-dev/sjtwo-c/-/blob/master/LPC408X_7X%20User%20Manual.pdf LPC408x/407x Datasheet]
*[https://www.freertos.org/Documentation/RTOS_book.html FreeRTOS Documentation]
*[http://elm-chan.org/fsw/ff/00index_e.html FatFs - Generic FAT Filesystem Module]
*[https://www.digi.com/resources/library/data-sheets/ds_xbeemultipointmodules Xbee Datasheet]
*[http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems Embedded Drivers & Real Time Operating Systems Lessons]

F21: Treasure Dive

2021-12-18T11:42:41Z

Proj user1: /* Implementation */

[[File:mermaid.png|thumb|500px|right|Game Screen Capture]]
[[File:Cmpe-244-controller-pic.png|thumb|600px|right|Game Controller]]
 
=== Grading Criteria ===

* How well is Software & Hardware Design described?
* How well can this report be used to reproduce this project?
* Code Quality
* Overall Report Quality:
** Software Block Diagrams
** Hardware Block Diagrams
**: Schematic Quality
** Quality of technical challenges and solutions adopted.


== '''Treasure Dive''' ==

== Abstract ==
Treasure Dive is a modern one player game inspired by the popular arcade game Breakout, which was released by Atari in 1976. The ancients rambled about an unconfirmed tale of a shipwreck that saw no survivors. As an adventurous and skilled free diver, you are seeking treasure buried deep in an ocean abyss. You dove down all the way to the seafloor and found the buried treasure but need to make it back to the surface to see another day! The player uses a wireless controller to control an on-screen paddle to break bricks on the top of the screen. When enough bricks are broken, a passage is revealed, allowing you to get closer to the surface.

== Objectives & Introduction ==
The objective of this project is to interface an LPC4088 to a VGA monitor by leveraging the LCD controller and use an accelerometer as the sensor for the player to control the paddle.
Two Zigbees RF modules will be integrated for wireless communication between the controller and LPC4088 board via UART protocol.
Digital audio output will be enabled by developing an I2S driver and porting an audio tracker library.
The General Purpose Direct Memory Access (GPDMA) controller memory-to-memory function will be utilized to decrease memory write times when updating the video buffer.
The DMA controller's memory-to-peripheral function will help reduce the frequent I2S interrupts when loading digital audio contents.
An SD card peripheral driver to facilitate memory exchange between an SD card will also be implemented.

=== Team Members & Responsibilities ===
* Brian Ho
** Wiki project schedule planning
** Interfacing the accelerometer
** Digital to Analog Conversion resistor ladder design used to interface digital LCD controller signal with analog VGA port signal
** PCB planning and acquisition
** Procurement of controller hardware
** Wireless communication via Zigbees RF modules and UART protocol
** Wireless controller logic and integration
** Ball movement and wall/brick collision logic
** Hardware/software development and integration
** Final testing and integration
* Billy Lai
** Wiki project schedule planning
** Increased CPU clock rate from 96 MHz to 120 MHz
** Initial memory write timing and measurements to estimate memory write capacity
** DMA driver memory-to-memory implementation for transferring graphics data to video buffer
** I2S with DMA memory-to-peripheral implementation to reduce memory-write interrupts when I2S Tx FIFO level is low
** Paddle movement and paddle collision logic, testing and integration
** Software development and integration
** Final testing and integration
* Jasdip Sekhon
** Wiki project schedule planning
** PCB design and planning
** File loading method algorithm for storing loaded files in memory
** Audio track and audio effects selection for gameplay audio
** Palette manipulation for gameplay
** Software development and integration
** Final testing and integration
* Isaac Wahhab
** Wiki project schedule planning
** DAC resistor ladder design and integration
** Hardware procurement (LPC4088 board, Zigbees, VGA monitor, etc)
** VGA driving through LCD controller
** Audio tracker library port
** Digital audio output using I2S
** Palette scrolling feature implementation
** Graphics rendering
** Vertical scrolling implementation for the video buffer
** Sprite testing and implementation for the ball
** SD card peripheral driver for reading assets from SD card
** Art assets acquisition
** Hardware/software development and integration
** Gameplay logic design
** Gameplay audio and level design
** Final testing and integration

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
*09/21/2021
|
*09/27/2021
|
* Read previous projects, gather information and discuss among the group members.
* Initial ordering of parts (LPC 4088 board, 64x64 LED screen)
|
* Completed
* Completed
|-
! scope="row"| 2
|
*09/28/2021
|
*10/04/2021
|
* Submit project proposals
|
* Completed
|-
! scope="row"| 3
|
*10/05/2021
|
*10/11/2021
|
* Test if board can drive VGA
* Test varying clock rates
* Hardware cursor on VGA
* Rough display driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 4
|
* 10/12/2021
|
* 10/18/2021
|
* DAC resistor ladder
* RGB channels
* Palette shifting
* Measure memory buffer writing timing
* Wiki schedule planning
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
*10/19/2021
|
*10/25/2021
|
* Calculate upper-bound for VGA refresh timings
* Create GitLab repository for project
* Joystick functionality
* SD card driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
*10/26/2021
|
*11/01/2021
|
* Audio peripheral
* Wireless communication using Zigbee module
* PCB planning
* Accelerometer driver
* Continue SD card driver
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/02/2021

|
* 11/08/2021
|
* Continue audio peripheral
* Continue wireless communication using Zigbee module
* Continue accelerometer driver
* PCB finalizing and ordering
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 11/09/2021
|
* 11/15/2021
|
* Convert PNG data to graphics format
* Game background functionality
* Sprite functionality
* User interface
* Integrate components
* Initial testing
|
* Completed
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 11/16/2021
|
* 11/22/2021
|
* Continue everything from last week
* Game physics (wall collision and AI movement)
* Measurements for wooden case
* Art assets (audio and visual)
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 10
|
*11/23/2021
|
*11/29/2021
|

* Fine-tune user experience
* Make case
|
* Completed
* Completed
|-
! scope="row"| 11
|
* 11/30/2021
|
* 12/06/2021
|
* Continue fine-tune user experience
* Integrate hardware
|
* Completed
* Completed
|-
! scope="row"| 12
|
* 12/07/2021
|
* 12/16/2021
|
* Final testing
* Final demo
|
* Completed
* Completed
|-
|}

== Bill Of Materials ==
{| class="wikitable" style="margin-left: 0px; margin-right: auto;"
|-
! scope="col"| Item#
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| [https://www.embeddedartists.com/products/lpc4088-developers-kit LPC4088-32 Developer's Kit]
| 1
| $300.00
|-
! scope="row"| 2
| Microcontroller Board
| [http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems/page/sj2-board SJ2 Board]
| 1
| $50.00
|-
! scope="row"| 3
| Bluetooth Module
| [https://hub.digi.com/support/products/digi-xbee/digi-xbee-s1 Digi XBee-S1]
| 2
| N/A
|-
! scope="row"| 4
| DAC PCB
| [https://jlcpcb.com JLCPCB (Set of 5)]
| 1
| $15.00
|-
! scope="row"| 4
| Monitor
| Asus TFT Monitor
| 1
| N/A
|-
! scope="row"| 5
| Mini Breadboard
| [https://www.amazon.com/EL-CK-002-Electronic-Breadboard-Capacitor-Potentiometer/dp/B01ERP6WL4/ref=pd_sbs_147_2/132-6936347-2117569?_encoding=UTF8&pd_rd_i=B01ERP6WL4&pd_rd_r=6a783095-ec87-42ae-9f01-a5ebc9a44607&pd_rd_w=V2aU6&pd_rd_wg=11h1i&pf_rd_p=d66372fe-68a6-48a3-90ec-41d7f64212be&pf_rd_r=368WNKXFBBQY2FJQ4119&psc=1&refRID=368WNKXFBBQY2FJQ4119 ELEGOO Mini Breadboard]
| 1
| $13.99
|-
! scope="row"| 6
| Arcade buttons
| [https://www.amazon.com/EG-STARTS-Illuminated-Buttons-Raspberry/dp/B01N11BDX9?pd_rd_w=DiSpt&pf_rd_p=5bc65336-a8b0-4888-9e70-4d4ce2f6bd59&pf_rd_r=JCH2VQYXFPHND4XZ4FAM&pd_rd_r=8d6243dd-e1bd-4dae-9b19-be2e00f3085f&pd_rd_wg=K4lt9&pd_rd_i=B01N11BDX9&psc=1&ref_=pd_bap_d_rp_1_i EG STARTS LED Arcade Buttons]
| 2
| $11.99
|-
! scope="row"| 7
| VGA cable
| N/A
| 1
| N/A
|-
! scope="row"| 7
| SD card
| N/A
| 1
| N/A
|-
|}

== Design & Implementation ==

=== Hardware Design ===
We designed our PCB using Autodesk EAGLE based on the tutorial on PCB design during the class lecture. We ordered our PCB to be manufactured from JLCPCB. The PCB consists of an R-2R resistor ladder DAC for the purpose of converting the digital signal from the microcontroller to analog to display graphics on the VGA monitor. Since the VGA input for each color channel is 0-0.7V, the resistor ladder scaled the LPC4088 outputs into proper DAC values for the VGA to take in. Five bits are passed to each of the red, green, and blue channels of the VGA in addition to other signals, such as Horizontal sync (HSYNC) and Vertical sync (VSYNC).
 
[[File:pcbSchematic.png|500px|thumb|left|PCB Schematic]]
[[File:pcbLayout.png|500px|thumb|right|PCB Layout]]
[[File:pcbPic-min.png|500px|thumb|right|Manufactured PCB]]
[[File:pcbManufactured.png|500px|thumb|left|Soldered PCB]]
 

=== Hardware Interface ===

Two Digi XBee devices were used to communicate between the SJ2-Board controller and the LPC4088 board to control the paddle on the game. The XBee devices were programmed using the XTCU programming application provided by Digi International. The SJ2-Board used one XBee as a transmitted and the LPC4088 used another as a receiver. The two modules communicated using UART frames and a single byte represented any change of movement or button presses sent by the controller.

{| class="wikitable"
|-
! scope="col"| XCTU
! scope="col"| Transmitter
! scope="col"| Receiver
|-
! scope="row"| CH
| C
| C
|-
! scope="row"| ID
| 6969
| 6969
|-
! scope="row"| CE
| Coordinator
| Endpoint
|-
! scope="row"| Baud Rate
| 9600
| 9600
|-
|}
[[File:sj2Board.png|380px|thumb|center|SJ2 Board]]
[[File:CMPE146_F16_MW_XbeeModule.jpg|200px|thumb|center|XBee Device]]
 

Two LED buttons were used to interface with the controller as inputs. They were both set up using GPIO.
[[File:VGA LED Button.jpg|200px|thumb|center|LED Button]]
 
[[File:VGA_Controller_Block_Diagram.PNG|380px|thumb|center|Controller Block Diagram]]
 

==== I2S Peripheral and Stereo Audio Decoder ====

The LPC4088 is equipped with an I2S stereo encoder. This module provides a hardware-enabled solution for sending I2S-encoded audio data packets to stereo decoders, which produce the corresponding audio waveform. For this project, since 16-bit stereo audio playback was chosen, data packets transmitted on this interface were sent in the following format:

[[File:I2S_audio_data_word.png]]

Here, each word of data encodes a single packet of audio in two 16-bit sections: one signal level for the left channel, and another for the right. These 4 bytes are sent at a rate of 44100 Hz, resulting in a total data transmission of 176400 bytes a second. Rather than needing to be software-defined, this transmission is automatically handled by hardware using a 8-word deep FIFO queue that manages streaming out the audio data at a fixed rate, saving a significant amount of CPU cycles.

This encoded audio signal was sent to a dedicated external [https://www.nxp.com/docs/en/data-sheet/UDA1380.pdf stereo decoder], which output to a 3.5 mm audio jack connected to speakers in the VGA monitor.

==== LCD Peripheral ====

The LCD peripheral was configured for 16-bit color palette (5 bits of color data for each color level -- red, green, and blue -- and an unused intensity bit), at a resolution of 800 by 600 pixels, and with an active color palette size of 256 colors. Since the logic for driving an LCD display has the same concepts (horizontal blanking, vertical blanking, etc) and timing information for any given resolution was the same, we were able to directly drive a VGA display with the same signals and timing data (after the digital signal had been converted to analog).

[[File:LCD_%2B_VGA_Video_signal.png]]

The timing register configurations needed to achieve the requested video display was derived from a dedicated website for [http://tinyvga.com/vga-timing/800x600@60Hz VGA signal info].

==== External Memory ====

The game's visual assets and tracker audio file (both after being loaded from an SD card) and the game's video buffer required more memory than what is present natively in the microcontroller. To accommodate this, an external 32MiB memory module was used. This module was interfaced with using the external memory controller peripheral present on the board.

=== Software Design ===
==== Controller ====

There is an internal accelerometer on the SJ2 board that is utilized as the movement controls for the paddle movement. The y-axis value of the magnitude is polled 17 times in 17 milliseconds and is divided by 17 to get an average reading per that time frame. The maximum tilt value that can be read by the controller is set to 700 out of the total 1000 (highest possible value). The current tilt value is divided by 31 to get a total of 32 possible readings to represent 5 bits of magnitude.

The controller sends one byte of data to the console whenever there is any change in magnitude on the accelerometer or button changes. The byte contains all the information needed to control the paddle on the screen. The first two bits on the byte represent 0 if there is no button input and 1 if there is a button input. The third bit represents whether the controller is tilted left or right with 0 being left and 1 being right. The last five bits represent the controller accelerometer's magnitude.

[[File:VGA_UART_Frame.PNG]]

==== Audio ====

Sound design was provided to the project (in the form of music and sound effects) through the use of a [https://https://en.wikipedia.org/wiki/Music_tracker tracker audio library]. Source code was adopted from the [https://github.com/schismtracker/schismtracker Schism Tracker] library and altered with the following modifications:

- All UI logic was stripped from the source code

- All file handling logic was reworked to use the SD card reading functions, as the project does not support POSIX file calls

- The [https://www.libsdl.org/ Simple DirectMedia Layer] hooks were replaced with a FreeRTOS task that periodically called the waveform generation callback function, which was passed to the I2S peripheral hooked up to a stereo decoder.

=== Implementation ===

==== Ball Logic ====

The ball is controlled by the buttons and the controller accelerometer values. When the button is pressed, the ball is initially sent with a random velocity. The velocity is determined by a substate timer and is initially set as one of 6 possible values.

{| class="wikitable"
|-
! scope="col"| Substate Count
! scope="col"| X-Velocity
! scope="col"| Y-Velocity
|-
! scope="row"| 0
| 4
| -2
|-
! scope="row"| 1
| -4
| -2
|-
! scope="row"| 2
| 2
| -4
|-
! scope="row"| 3
| -2
| -4
|-
! scope="row"| 4
| 1
| -5
|-
! scope="row"| 5
| -1
| -5
|-
|}

The ball's velocity changes whenever it detects a collision with either the side of the screen, the bricks on the top of the level, or the paddle at the bottom. If colliding with the paddle, the ball's velocity will take into account the paddle's velocity and add it to its current velocity. This is done by adding the paddle's x-velocity and adjusting the y-velocity by normalizing the velocity vectors to have a magnitude of 5. This means the ball will always be moving with a magnitude of 5 in any direction after it hits the paddle. If the ball collides with anything other than the paddle, then the velocity of the ball is changed with no increase or decrease of magnitude. If the ball collides on the side of a wall or block, the x-velocity will reflect (go from negative to positive or vice-versa) and the y-velocity will remain constant. If the ball collides on the top or bottom of a block, the y-velocity will reflect and the x-velocity will remain constant. If it appears that the ball collides with the side and top or bottom of something, then the ball will check whichever face it will collide with first and then calculate the other face. Lastly, when the ball collides with the bottom of the screen, then the player experiences a "death" and the ball will reset on the paddle. If the ball collides with the top of the screen, the screen will scroll up for the user to play on the next level.

Whenever the ball collides with a brick, the brick will disappear. To accomplish this, there is a 2-D array of unsigned integers that contains the values of the bricks. If the integer is 0, then there is no block. If the integer is any value other than 0, then the block is filled and has collision logic. If the ball approaches a region that might have a block, then there will be a check within that 2-D to see if the block is filled or empty. If the block is filled, there will be the collision logic that changes the vector of the ball and sets the integer value in the 2-D array to 0 to represent that the block has been broken.

==== Paddle Movement ====
The paddle movement is controlled by the accelerometer data being wirelessly transmitted from the SJ-Two board to the LPC4088 board. The accelerometer data will map to 63 possible indexes for the paddle location. Originally, the paddle would jump directly into the corresponding index location depending on the accelerometer input. We later iterated on this process by implementing an algorithm to smoothly transition into these indexes. This was done by calculating a paddle velocity using the "current desired paddle position" and the previous paddle position. We adjusted the parameters of the algorithm iteratively to produce the smooth paddle movement.

==== Paddle Collision Logic ====
The position of the ball is calculated by adding delta_x and delta_y to the ball's current x and y position. For each frame, if the ball's next Y position crosses the paddle's Y position, we would check for a paddle collision. To do this, we used the delta_x and delta_y to get the ball's movement as a slope. We then substituted the slope, the ball's current position, and 1 pixel above the paddle's Y position into a point-slope equation (y2 - y1) = m(x2 - x1) to get the paddle's X position at the time of the ball reaching the pixel above paddle's Y position. After finding the location where the ball is expected to collide with the paddle, we then calculated the ball's collision magnitude and the ball's next location's magnitude. Using a ratio of these two magnitudes, we have a frame time fraction of when the ball possibly collided with the paddle. Then, by using the paddle's current position and paddle velocity, we are able to find where the paddle previously was, and measure how far the paddle had moved during that time frame. By multiplying the time frame from the ball's collision magnitude to the paddle's change in X position, we are able to find whether the paddle was under the ball at the time of the ball reaching the pixel above the paddle's Y position.

== Testing & Technical Challenges ==

=== VGA Interfacing ===
A DAC resistor ladder was designed and a corresponding custom PCB was acquired to interface the 3.3V digital LCD controller signal with the analog VGA input signal. The VGA cable takes between 0V (completely dark) and 0.7V (maximum brightness) for color pin inputs. The DAC resistor ladder was designed and implemented on a breadboard to verify the design. Validating the design on a breadboard gave us a clear vision during PCB design and planning.

=== VGA Write Timings ===
There were initial tests that were required to be done to see if using a VGA monitor as an output was a viable option for this project. We had to test memory write timings to see if they were fast enough to match the refresh rate of the monitor. Those tests would decide how much of the screen buffer we could write to during the blanking periods during the VGA refresh. This was a good learning experience for testing hardware capabilities and should be done whenever introducing any new aspects to a project.

=== Ball Collision Logic ===
The initial version of the ball collision used pixel values to see if the ball was entering the brick. The problem was that this logic did not account for the case where the ball enters the brick diagonally and the logic for that version of the collision had to be scrapped. The later iteration of the logic was changed to checking whether the ball was potentially crossing into an area that had a brick. This was done by separating the checks into a grid that represented the locations of the blocks. In this case, there was an initial check to see if the ball was crossing a boundary where there might be a new brick. If the ball is crossing the x-boundary and not the y-boundary, then there would be a collision check to see if it collided with a block. If it is crossing the y-boundary and not the x-boundary, once again then there would be a collision check to see if it collided with a block. If it is crossing both the x-boundary and y-boundary, then there would be a check to see which boundary the ball would cross first. This new logic allowed for us to account for the case where the block enters the block diagonally. This taught us that sometimes it is beneficial to scrap a previous iteration and use a new approach to solving a problem because it may lead to better results.

=== Paddle Movement ===
One issue we discovered was the paddle would slow down as it approached the walls. This was due to the difference in "current desired paddle position" and the previous paddle position becoming smaller as the walls were approached, leading to a smaller paddle velocity. To account for this, whenever the accelerometer sent data that mapped to the minimum or maximum paddle index, we would increase the distance of the "current desired paddle position" so the paddle velocity would not slow the paddle down as it approached the walls.

=== Paddle Collision Logic ===
During final integration and testing, one of the biggest issues we encountered was a bug where the ball would not detect a collision with the paddle. The ball would phase right through the paddle instead of bouncing off of it. One of the great ideas that we thought of was to change the color of the ball during each check to see if the checks would go through. This debug check allowed us to verify that section of code that checked for paddle collisions was being skipped even though the ball's next Y position had crossed the paddle's Y position. By having this debugging method, we were able to root cause the issue.

== Conclusion ==
Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

=== Project Video ===
[https://www.youtube.com/watch?v=ipngAGlN86c SJSU CMPE 244 FALL 2021: Embedded Software Project Gameplay]

=== Project Source Code ===
* [https://gitlab.com/fayek.wahhab/sjtwo-c/-/tree/feature/i2s_dma2 Treasure Dive Baseline for Demo Source Code]
* [https://gitlab.com/hobriandh/sjtwo-c/-/tree/final_controller Controller for Demo Source Code]

== References ==
=== Acknowledgement ===
We would like to acknowledge and thank our instructor Preetpal Kang for sharing his knowledge in embedded software development using FreeRTOS.

=== Appendix ===
*[https://gitlab.com/sjtwo-c-dev/sjtwo-c/-/blob/master/LPC408X_7X%20User%20Manual.pdf LPC408x/407x Datasheet]
*[https://www.freertos.org/Documentation/RTOS_book.html FreeRTOS Documentation]
*[http://elm-chan.org/fsw/ff/00index_e.html FatFs - Generic FAT Filesystem Module]
*[https://www.digi.com/resources/library/data-sheets/ds_xbeemultipointmodules Xbee Datasheet]
*[http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems Embedded Drivers & Real Time Operating Systems Lessons]

F21: Treasure Dive

2021-12-18T11:36:41Z

Proj user1: /* Testing & Technical Challenges */

[[File:mermaid.png|thumb|500px|right|Game Screen Capture]]
[[File:Cmpe-244-controller-pic.png|thumb|600px|right|Game Controller]]
 
=== Grading Criteria ===

* How well is Software & Hardware Design described?
* How well can this report be used to reproduce this project?
* Code Quality
* Overall Report Quality:
** Software Block Diagrams
** Hardware Block Diagrams
**: Schematic Quality
** Quality of technical challenges and solutions adopted.


== '''Treasure Dive''' ==

== Abstract ==
Treasure Dive is a modern one player game inspired by the popular arcade game Breakout, which was released by Atari in 1976. The ancients rambled about an unconfirmed tale of a shipwreck that saw no survivors. As an adventurous and skilled free diver, you are seeking treasure buried deep in an ocean abyss. You dove down all the way to the seafloor and found the buried treasure but need to make it back to the surface to see another day! The player uses a wireless controller to control an on-screen paddle to break bricks on the top of the screen. When enough bricks are broken, a passage is revealed, allowing you to get closer to the surface.

== Objectives & Introduction ==
The objective of this project is to interface an LPC4088 to a VGA monitor by leveraging the LCD controller and use an accelerometer as the sensor for the player to control the paddle.
Two Zigbees RF modules will be integrated for wireless communication between the controller and LPC4088 board via UART protocol.
Digital audio output will be enabled by developing an I2S driver and porting an audio tracker library.
The General Purpose Direct Memory Access (GPDMA) controller memory-to-memory function will be utilized to decrease memory write times when updating the video buffer.
The DMA controller's memory-to-peripheral function will help reduce the frequent I2S interrupts when loading digital audio contents.
An SD card peripheral driver to facilitate memory exchange between an SD card will also be implemented.

=== Team Members & Responsibilities ===
* Brian Ho
** Wiki project schedule planning
** Interfacing the accelerometer
** Digital to Analog Conversion resistor ladder design used to interface digital LCD controller signal with analog VGA port signal
** PCB planning and acquisition
** Procurement of controller hardware
** Wireless communication via Zigbees RF modules and UART protocol
** Wireless controller logic and integration
** Ball movement and wall/brick collision logic
** Hardware/software development and integration
** Final testing and integration
* Billy Lai
** Wiki project schedule planning
** Increased CPU clock rate from 96 MHz to 120 MHz
** Initial memory write timing and measurements to estimate memory write capacity
** DMA driver memory-to-memory implementation for transferring graphics data to video buffer
** I2S with DMA memory-to-peripheral implementation to reduce memory-write interrupts when I2S Tx FIFO level is low
** Paddle movement and paddle collision logic, testing and integration
** Software development and integration
** Final testing and integration
* Jasdip Sekhon
** Wiki project schedule planning
** PCB design and planning
** File loading method algorithm for storing loaded files in memory
** Audio track and audio effects selection for gameplay audio
** Palette manipulation for gameplay
** Software development and integration
** Final testing and integration
* Isaac Wahhab
** Wiki project schedule planning
** DAC resistor ladder design and integration
** Hardware procurement (LPC4088 board, Zigbees, VGA monitor, etc)
** VGA driving through LCD controller
** Audio tracker library port
** Digital audio output using I2S
** Palette scrolling feature implementation
** Graphics rendering
** Vertical scrolling implementation for the video buffer
** Sprite testing and implementation for the ball
** SD card peripheral driver for reading assets from SD card
** Art assets acquisition
** Hardware/software development and integration
** Gameplay logic design
** Gameplay audio and level design
** Final testing and integration

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
*09/21/2021
|
*09/27/2021
|
* Read previous projects, gather information and discuss among the group members.
* Initial ordering of parts (LPC 4088 board, 64x64 LED screen)
|
* Completed
* Completed
|-
! scope="row"| 2
|
*09/28/2021
|
*10/04/2021
|
* Submit project proposals
|
* Completed
|-
! scope="row"| 3
|
*10/05/2021
|
*10/11/2021
|
* Test if board can drive VGA
* Test varying clock rates
* Hardware cursor on VGA
* Rough display driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 4
|
* 10/12/2021
|
* 10/18/2021
|
* DAC resistor ladder
* RGB channels
* Palette shifting
* Measure memory buffer writing timing
* Wiki schedule planning
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
*10/19/2021
|
*10/25/2021
|
* Calculate upper-bound for VGA refresh timings
* Create GitLab repository for project
* Joystick functionality
* SD card driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
*10/26/2021
|
*11/01/2021
|
* Audio peripheral
* Wireless communication using Zigbee module
* PCB planning
* Accelerometer driver
* Continue SD card driver
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/02/2021

|
* 11/08/2021
|
* Continue audio peripheral
* Continue wireless communication using Zigbee module
* Continue accelerometer driver
* PCB finalizing and ordering
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 11/09/2021
|
* 11/15/2021
|
* Convert PNG data to graphics format
* Game background functionality
* Sprite functionality
* User interface
* Integrate components
* Initial testing
|
* Completed
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 11/16/2021
|
* 11/22/2021
|
* Continue everything from last week
* Game physics (wall collision and AI movement)
* Measurements for wooden case
* Art assets (audio and visual)
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 10
|
*11/23/2021
|
*11/29/2021
|

* Fine-tune user experience
* Make case
|
* Completed
* Completed
|-
! scope="row"| 11
|
* 11/30/2021
|
* 12/06/2021
|
* Continue fine-tune user experience
* Integrate hardware
|
* Completed
* Completed
|-
! scope="row"| 12
|
* 12/07/2021
|
* 12/16/2021
|
* Final testing
* Final demo
|
* Completed
* Completed
|-
|}

== Bill Of Materials ==
{| class="wikitable" style="margin-left: 0px; margin-right: auto;"
|-
! scope="col"| Item#
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| [https://www.embeddedartists.com/products/lpc4088-developers-kit LPC4088-32 Developer's Kit]
| 1
| $300.00
|-
! scope="row"| 2
| Microcontroller Board
| [http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems/page/sj2-board SJ2 Board]
| 1
| $50.00
|-
! scope="row"| 3
| Bluetooth Module
| [https://hub.digi.com/support/products/digi-xbee/digi-xbee-s1 Digi XBee-S1]
| 2
| N/A
|-
! scope="row"| 4
| DAC PCB
| [https://jlcpcb.com JLCPCB (Set of 5)]
| 1
| $15.00
|-
! scope="row"| 4
| Monitor
| Asus TFT Monitor
| 1
| N/A
|-
! scope="row"| 5
| Mini Breadboard
| [https://www.amazon.com/EL-CK-002-Electronic-Breadboard-Capacitor-Potentiometer/dp/B01ERP6WL4/ref=pd_sbs_147_2/132-6936347-2117569?_encoding=UTF8&pd_rd_i=B01ERP6WL4&pd_rd_r=6a783095-ec87-42ae-9f01-a5ebc9a44607&pd_rd_w=V2aU6&pd_rd_wg=11h1i&pf_rd_p=d66372fe-68a6-48a3-90ec-41d7f64212be&pf_rd_r=368WNKXFBBQY2FJQ4119&psc=1&refRID=368WNKXFBBQY2FJQ4119 ELEGOO Mini Breadboard]
| 1
| $13.99
|-
! scope="row"| 6
| Arcade buttons
| [https://www.amazon.com/EG-STARTS-Illuminated-Buttons-Raspberry/dp/B01N11BDX9?pd_rd_w=DiSpt&pf_rd_p=5bc65336-a8b0-4888-9e70-4d4ce2f6bd59&pf_rd_r=JCH2VQYXFPHND4XZ4FAM&pd_rd_r=8d6243dd-e1bd-4dae-9b19-be2e00f3085f&pd_rd_wg=K4lt9&pd_rd_i=B01N11BDX9&psc=1&ref_=pd_bap_d_rp_1_i EG STARTS LED Arcade Buttons]
| 2
| $11.99
|-
! scope="row"| 7
| VGA cable
| N/A
| 1
| N/A
|-
! scope="row"| 7
| SD card
| N/A
| 1
| N/A
|-
|}

== Design & Implementation ==

=== Hardware Design ===
We designed our PCB using Autodesk EAGLE based on the tutorial on PCB design during the class lecture. We ordered our PCB to be manufactured from JLCPCB. The PCB consists of an R-2R resistor ladder DAC for the purpose of converting the digital signal from the microcontroller to analog to display graphics on the VGA monitor. Since the VGA input for each color channel is 0-0.7V, the resistor ladder scaled the LPC4088 outputs into proper DAC values for the VGA to take in. Five bits are passed to each of the red, green, and blue channels of the VGA in addition to other signals, such as Horizontal sync (HSYNC) and Vertical sync (VSYNC).
 
[[File:pcbSchematic.png|500px|thumb|left|PCB Schematic]]
[[File:pcbLayout.png|500px|thumb|right|PCB Layout]]
[[File:pcbPic-min.png|500px|thumb|right|Manufactured PCB]]
[[File:pcbManufactured.png|500px|thumb|left|Soldered PCB]]
 

=== Hardware Interface ===

Two Digi XBee devices were used to communicate between the SJ2-Board controller and the LPC4088 board to control the paddle on the game. The XBee devices were programmed using the XTCU programming application provided by Digi International. The SJ2-Board used one XBee as a transmitted and the LPC4088 used another as a receiver. The two modules communicated using UART frames and a single byte represented any change of movement or button presses sent by the controller.

{| class="wikitable"
|-
! scope="col"| XCTU
! scope="col"| Transmitter
! scope="col"| Receiver
|-
! scope="row"| CH
| C
| C
|-
! scope="row"| ID
| 6969
| 6969
|-
! scope="row"| CE
| Coordinator
| Endpoint
|-
! scope="row"| Baud Rate
| 9600
| 9600
|-
|}
[[File:sj2Board.png|380px|thumb|center|SJ2 Board]]
[[File:CMPE146_F16_MW_XbeeModule.jpg|200px|thumb|center|XBee Device]]
 

Two LED buttons were used to interface with the controller as inputs. They were both set up using GPIO.
[[File:VGA LED Button.jpg|200px|thumb|center|LED Button]]
 
[[File:VGA_Controller_Block_Diagram.PNG|380px|thumb|center|Controller Block Diagram]]
 

==== I2S Peripheral and Stereo Audio Decoder ====

The LPC4088 is equipped with an I2S stereo encoder. This module provides a hardware-enabled solution for sending I2S-encoded audio data packets to stereo decoders, which produce the corresponding audio waveform. For this project, since 16-bit stereo audio playback was chosen, data packets transmitted on this interface were sent in the following format:

[[File:I2S_audio_data_word.png]]

Here, each word of data encodes a single packet of audio in two 16-bit sections: one signal level for the left channel, and another for the right. These 4 bytes are sent at a rate of 44100 Hz, resulting in a total data transmission of 176400 bytes a second. Rather than needing to be software-defined, this transmission is automatically handled by hardware using a 8-word deep FIFO queue that manages streaming out the audio data at a fixed rate, saving a significant amount of CPU cycles.

This encoded audio signal was sent to a dedicated external [https://www.nxp.com/docs/en/data-sheet/UDA1380.pdf stereo decoder], which output to a 3.5 mm audio jack connected to speakers in the VGA monitor.

==== LCD Peripheral ====

The LCD peripheral was configured for 16-bit color palette (5 bits of color data for each color level -- red, green, and blue -- and an unused intensity bit), at a resolution of 800 by 600 pixels, and with an active color palette size of 256 colors. Since the logic for driving an LCD display has the same concepts (horizontal blanking, vertical blanking, etc) and timing information for any given resolution was the same, we were able to directly drive a VGA display with the same signals and timing data (after the digital signal had been converted to analog).

[[File:LCD_%2B_VGA_Video_signal.png]]

The timing register configurations needed to achieve the requested video display was derived from a dedicated website for [http://tinyvga.com/vga-timing/800x600@60Hz VGA signal info].

==== External Memory ====

The game's visual assets and tracker audio file (both after being loaded from an SD card) and the game's video buffer required more memory than what is present natively in the microcontroller. To accommodate this, an external 32MiB memory module was used. This module was interfaced with using the external memory controller peripheral present on the board.

=== Software Design ===
==== Controller ====

There is an internal accelerometer on the SJ2 board that is utilized as the movement controls for the paddle movement. The y-axis value of the magnitude is polled 17 times in 17 milliseconds and is divided by 17 to get an average reading per that time frame. The maximum tilt value that can be read by the controller is set to 700 out of the total 1000 (highest possible value). The current tilt value is divided by 31 to get a total of 32 possible readings to represent 5 bits of magnitude.

The controller sends one byte of data to the console whenever there is any change in magnitude on the accelerometer or button changes. The byte contains all the information needed to control the paddle on the screen. The first two bits on the byte represent 0 if there is no button input and 1 if there is a button input. The third bit represents whether the controller is tilted left or right with 0 being left and 1 being right. The last five bits represent the controller accelerometer's magnitude.

[[File:VGA_UART_Frame.PNG]]

==== Audio ====

Sound design was provided to the project (in the form of music and sound effects) through the use of a [https://https://en.wikipedia.org/wiki/Music_tracker tracker audio library]. Source code was adopted from the [https://github.com/schismtracker/schismtracker Schism Tracker] library and altered with the following modifications:

- All UI logic was stripped from the source code

- All file handling logic was reworked to use the SD card reading functions, as the project does not support POSIX file calls

- The [https://www.libsdl.org/ Simple DirectMedia Layer] hooks were replaced with a FreeRTOS task that periodically called the waveform generation callback function, which was passed to the I2S peripheral hooked up to a stereo decoder.

=== Implementation ===

==== Ball Logic ====

The ball is controlled by the buttons and the controller accelerometer values. When the button is pressed, the ball is initially sent with a random velocity. The velocity is determined by a substate timer and is initially set as one of 6 possible values.

{| class="wikitable"
|-
! scope="col"| Substate Count
! scope="col"| X-Velocity
! scope="col"| Y-Velocity
|-
! scope="row"| 0
| 4
| -2
|-
! scope="row"| 1
| -4
| -2
|-
! scope="row"| 2
| 2
| -4
|-
! scope="row"| 3
| -2
| -4
|-
! scope="row"| 4
| 1
| -5
|-
! scope="row"| 5
| -1
| -5
|-
|}

The ball's velocity changes whenever it detects a collision with either the side of the screen, the bricks on the top of the level, or the paddle at the bottom. If colliding with the paddle, the ball's velocity will take into account the paddle's velocity and add it to its current velocity. This is done by adding the paddle's x-velocity and adjusting the y-velocity by normalizing the velocity vectors to have a magnitude of 5. This means the ball will always be moving with a magnitude of 5 in any direction after it hits the paddle. If the ball collides with anything other than the paddle, then the velocity of the ball is changed with no increase or decrease of magnitude. If the ball collides on the side of a wall or block, the x-velocity will reflect (go from negative to positive or vice-versa) and the y-velocity will remain constant. If the ball collides on the top or bottom of a block, the y-velocity will reflect and the x-velocity will remain constant. If it appears that the ball collides with the side and top or bottom of something, then the ball will check whichever face it will collide with first and then calculate the other face. Lastly, when the ball collides with the bottom of the screen, then the player experiences a "death" and the ball will reset on the paddle. If the ball collides with the top of the screen, the screen will scroll up for the user to play on the next level.

Whenever the ball collides with a brick, the brick will disappear. To accomplish this, there is a 2-D array of unsigned integers that contains the values of the bricks. If the integer is 0, then there is no block. If the integer is any value other than 0, then the block is filled and has collision logic. If the ball approaches a region that might have a block, then there will be a check within that 2-D to see if the block is filled or empty. If the block is filled, there will be the collision logic that changes the vector of the ball and sets the integer value in the 2-D array to 0 to represent that the block has been broken.

==== Paddle Movement ====
The paddle movement is controlled by the accelerometer data being wirelessly transmitted from the SJ-Two board to the LPC4088 board. The accelerometer data will map to 63 possible indexes for the paddle location. Originally, the paddle would jump directly into the corresponding index location depending on the accelerometer input. We later iterated on this process by implementing an algorithm to smoothly transition into these indexes. This was done by calculating a paddle velocity using the "current desired paddle position" and the previous paddle position. We adjusted the parameters of the algorithm iteratively to produce the smooth paddle movement.

== Testing & Technical Challenges ==

=== VGA Interfacing ===
A DAC resistor ladder was designed and a corresponding custom PCB was acquired to interface the 3.3V digital LCD controller signal with the analog VGA input signal. The VGA cable takes between 0V (completely dark) and 0.7V (maximum brightness) for color pin inputs. The DAC resistor ladder was designed and implemented on a breadboard to verify the design. Validating the design on a breadboard gave us a clear vision during PCB design and planning.

=== VGA Write Timings ===
There were initial tests that were required to be done to see if using a VGA monitor as an output was a viable option for this project. We had to test memory write timings to see if they were fast enough to match the refresh rate of the monitor. Those tests would decide how much of the screen buffer we could write to during the blanking periods during the VGA refresh. This was a good learning experience for testing hardware capabilities and should be done whenever introducing any new aspects to a project.

=== Ball Collision Logic ===
The initial version of the ball collision used pixel values to see if the ball was entering the brick. The problem was that this logic did not account for the case where the ball enters the brick diagonally and the logic for that version of the collision had to be scrapped. The later iteration of the logic was changed to checking whether the ball was potentially crossing into an area that had a brick. This was done by separating the checks into a grid that represented the locations of the blocks. In this case, there was an initial check to see if the ball was crossing a boundary where there might be a new brick. If the ball is crossing the x-boundary and not the y-boundary, then there would be a collision check to see if it collided with a block. If it is crossing the y-boundary and not the x-boundary, once again then there would be a collision check to see if it collided with a block. If it is crossing both the x-boundary and y-boundary, then there would be a check to see which boundary the ball would cross first. This new logic allowed for us to account for the case where the block enters the block diagonally. This taught us that sometimes it is beneficial to scrap a previous iteration and use a new approach to solving a problem because it may lead to better results.

=== Paddle Movement ===
One issue we discovered was the paddle would slow down as it approached the walls. This was due to the difference in "current desired paddle position" and the previous paddle position becoming smaller as the walls were approached, leading to a smaller paddle velocity. To account for this, whenever the accelerometer sent data that mapped to the minimum or maximum paddle index, we would increase the distance of the "current desired paddle position" so the paddle velocity would not slow the paddle down as it approached the walls.

=== Paddle Collision Logic ===
During final integration and testing, one of the biggest issues we encountered was a bug where the ball would not detect a collision with the paddle. The ball would phase right through the paddle instead of bouncing off of it. One of the great ideas that we thought of was to change the color of the ball during each check to see if the checks would go through. This debug check allowed us to verify that section of code that checked for paddle collisions was being skipped even though the ball's next Y position had crossed the paddle's Y position. By having this debugging method, we were able to root cause the issue.

== Conclusion ==
Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

=== Project Video ===
[https://www.youtube.com/watch?v=ipngAGlN86c SJSU CMPE 244 FALL 2021: Embedded Software Project Gameplay]

=== Project Source Code ===
* [https://gitlab.com/fayek.wahhab/sjtwo-c/-/tree/feature/i2s_dma2 Treasure Dive Baseline for Demo Source Code]
* [https://gitlab.com/hobriandh/sjtwo-c/-/tree/final_controller Controller for Demo Source Code]

== References ==
=== Acknowledgement ===
We would like to acknowledge and thank our instructor Preetpal Kang for sharing his knowledge in embedded software development using FreeRTOS.

=== Appendix ===
*[https://gitlab.com/sjtwo-c-dev/sjtwo-c/-/blob/master/LPC408X_7X%20User%20Manual.pdf LPC408x/407x Datasheet]
*[https://www.freertos.org/Documentation/RTOS_book.html FreeRTOS Documentation]
*[http://elm-chan.org/fsw/ff/00index_e.html FatFs - Generic FAT Filesystem Module]
*[https://www.digi.com/resources/library/data-sheets/ds_xbeemultipointmodules Xbee Datasheet]
*[http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems Embedded Drivers & Real Time Operating Systems Lessons]

F21: Treasure Dive

2021-12-18T11:36:04Z

Proj user1: /* LCD Peripheral */

[[File:mermaid.png|thumb|500px|right|Game Screen Capture]]
[[File:Cmpe-244-controller-pic.png|thumb|600px|right|Game Controller]]
 
=== Grading Criteria ===

* How well is Software & Hardware Design described?
* How well can this report be used to reproduce this project?
* Code Quality
* Overall Report Quality:
** Software Block Diagrams
** Hardware Block Diagrams
**: Schematic Quality
** Quality of technical challenges and solutions adopted.


== '''Treasure Dive''' ==

== Abstract ==
Treasure Dive is a modern one player game inspired by the popular arcade game Breakout, which was released by Atari in 1976. The ancients rambled about an unconfirmed tale of a shipwreck that saw no survivors. As an adventurous and skilled free diver, you are seeking treasure buried deep in an ocean abyss. You dove down all the way to the seafloor and found the buried treasure but need to make it back to the surface to see another day! The player uses a wireless controller to control an on-screen paddle to break bricks on the top of the screen. When enough bricks are broken, a passage is revealed, allowing you to get closer to the surface.

== Objectives & Introduction ==
The objective of this project is to interface an LPC4088 to a VGA monitor by leveraging the LCD controller and use an accelerometer as the sensor for the player to control the paddle.
Two Zigbees RF modules will be integrated for wireless communication between the controller and LPC4088 board via UART protocol.
Digital audio output will be enabled by developing an I2S driver and porting an audio tracker library.
The General Purpose Direct Memory Access (GPDMA) controller memory-to-memory function will be utilized to decrease memory write times when updating the video buffer.
The DMA controller's memory-to-peripheral function will help reduce the frequent I2S interrupts when loading digital audio contents.
An SD card peripheral driver to facilitate memory exchange between an SD card will also be implemented.

=== Team Members & Responsibilities ===
* Brian Ho
** Wiki project schedule planning
** Interfacing the accelerometer
** Digital to Analog Conversion resistor ladder design used to interface digital LCD controller signal with analog VGA port signal
** PCB planning and acquisition
** Procurement of controller hardware
** Wireless communication via Zigbees RF modules and UART protocol
** Wireless controller logic and integration
** Ball movement and wall/brick collision logic
** Hardware/software development and integration
** Final testing and integration
* Billy Lai
** Wiki project schedule planning
** Increased CPU clock rate from 96 MHz to 120 MHz
** Initial memory write timing and measurements to estimate memory write capacity
** DMA driver memory-to-memory implementation for transferring graphics data to video buffer
** I2S with DMA memory-to-peripheral implementation to reduce memory-write interrupts when I2S Tx FIFO level is low
** Paddle movement and paddle collision logic, testing and integration
** Software development and integration
** Final testing and integration
* Jasdip Sekhon
** Wiki project schedule planning
** PCB design and planning
** File loading method algorithm for storing loaded files in memory
** Audio track and audio effects selection for gameplay audio
** Palette manipulation for gameplay
** Software development and integration
** Final testing and integration
* Isaac Wahhab
** Wiki project schedule planning
** DAC resistor ladder design and integration
** Hardware procurement (LPC4088 board, Zigbees, VGA monitor, etc)
** VGA driving through LCD controller
** Audio tracker library port
** Digital audio output using I2S
** Palette scrolling feature implementation
** Graphics rendering
** Vertical scrolling implementation for the video buffer
** Sprite testing and implementation for the ball
** SD card peripheral driver for reading assets from SD card
** Art assets acquisition
** Hardware/software development and integration
** Gameplay logic design
** Gameplay audio and level design
** Final testing and integration

== Schedule ==

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Status
|-
! scope="row"| 1
|
*09/21/2021
|
*09/27/2021
|
* Read previous projects, gather information and discuss among the group members.
* Initial ordering of parts (LPC 4088 board, 64x64 LED screen)
|
* Completed
* Completed
|-
! scope="row"| 2
|
*09/28/2021
|
*10/04/2021
|
* Submit project proposals
|
* Completed
|-
! scope="row"| 3
|
*10/05/2021
|
*10/11/2021
|
* Test if board can drive VGA
* Test varying clock rates
* Hardware cursor on VGA
* Rough display driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 4
|
* 10/12/2021
|
* 10/18/2021
|
* DAC resistor ladder
* RGB channels
* Palette shifting
* Measure memory buffer writing timing
* Wiki schedule planning
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 5
|
*10/19/2021
|
*10/25/2021
|
* Calculate upper-bound for VGA refresh timings
* Create GitLab repository for project
* Joystick functionality
* SD card driver
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 6
|
*10/26/2021
|
*11/01/2021
|
* Audio peripheral
* Wireless communication using Zigbee module
* PCB planning
* Accelerometer driver
* Continue SD card driver
|
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 7
|
* 11/02/2021

|
* 11/08/2021
|
* Continue audio peripheral
* Continue wireless communication using Zigbee module
* Continue accelerometer driver
* PCB finalizing and ordering
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 8
|
* 11/09/2021
|
* 11/15/2021
|
* Convert PNG data to graphics format
* Game background functionality
* Sprite functionality
* User interface
* Integrate components
* Initial testing
|
* Completed
* Completed
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 9
|
* 11/16/2021
|
* 11/22/2021
|
* Continue everything from last week
* Game physics (wall collision and AI movement)
* Measurements for wooden case
* Art assets (audio and visual)
|
* Completed
* Completed
* Completed
* Completed
|-
! scope="row"| 10
|
*11/23/2021
|
*11/29/2021
|

* Fine-tune user experience
* Make case
|
* Completed
* Completed
|-
! scope="row"| 11
|
* 11/30/2021
|
* 12/06/2021
|
* Continue fine-tune user experience
* Integrate hardware
|
* Completed
* Completed
|-
! scope="row"| 12
|
* 12/07/2021
|
* 12/16/2021
|
* Final testing
* Final demo
|
* Completed
* Completed
|-
|}

== Bill Of Materials ==
{| class="wikitable" style="margin-left: 0px; margin-right: auto;"
|-
! scope="col"| Item#
! scope="col"| Part Description
! scope="col"| Part Model & Vendor
! scope="col"| Quantity
! scope="col"| Cost
|-
! scope="row"| 1
| Microcontroller Board
| [https://www.embeddedartists.com/products/lpc4088-developers-kit LPC4088-32 Developer's Kit]
| 1
| $300.00
|-
! scope="row"| 2
| Microcontroller Board
| [http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems/page/sj2-board SJ2 Board]
| 1
| $50.00
|-
! scope="row"| 3
| Bluetooth Module
| [https://hub.digi.com/support/products/digi-xbee/digi-xbee-s1 Digi XBee-S1]
| 2
| N/A
|-
! scope="row"| 4
| DAC PCB
| [https://jlcpcb.com JLCPCB (Set of 5)]
| 1
| $15.00
|-
! scope="row"| 4
| Monitor
| Asus TFT Monitor
| 1
| N/A
|-
! scope="row"| 5
| Mini Breadboard
| [https://www.amazon.com/EL-CK-002-Electronic-Breadboard-Capacitor-Potentiometer/dp/B01ERP6WL4/ref=pd_sbs_147_2/132-6936347-2117569?_encoding=UTF8&pd_rd_i=B01ERP6WL4&pd_rd_r=6a783095-ec87-42ae-9f01-a5ebc9a44607&pd_rd_w=V2aU6&pd_rd_wg=11h1i&pf_rd_p=d66372fe-68a6-48a3-90ec-41d7f64212be&pf_rd_r=368WNKXFBBQY2FJQ4119&psc=1&refRID=368WNKXFBBQY2FJQ4119 ELEGOO Mini Breadboard]
| 1
| $13.99
|-
! scope="row"| 6
| Arcade buttons
| [https://www.amazon.com/EG-STARTS-Illuminated-Buttons-Raspberry/dp/B01N11BDX9?pd_rd_w=DiSpt&pf_rd_p=5bc65336-a8b0-4888-9e70-4d4ce2f6bd59&pf_rd_r=JCH2VQYXFPHND4XZ4FAM&pd_rd_r=8d6243dd-e1bd-4dae-9b19-be2e00f3085f&pd_rd_wg=K4lt9&pd_rd_i=B01N11BDX9&psc=1&ref_=pd_bap_d_rp_1_i EG STARTS LED Arcade Buttons]
| 2
| $11.99
|-
! scope="row"| 7
| VGA cable
| N/A
| 1
| N/A
|-
! scope="row"| 7
| SD card
| N/A
| 1
| N/A
|-
|}

== Design & Implementation ==

=== Hardware Design ===
We designed our PCB using Autodesk EAGLE based on the tutorial on PCB design during the class lecture. We ordered our PCB to be manufactured from JLCPCB. The PCB consists of an R-2R resistor ladder DAC for the purpose of converting the digital signal from the microcontroller to analog to display graphics on the VGA monitor. Since the VGA input for each color channel is 0-0.7V, the resistor ladder scaled the LPC4088 outputs into proper DAC values for the VGA to take in. Five bits are passed to each of the red, green, and blue channels of the VGA in addition to other signals, such as Horizontal sync (HSYNC) and Vertical sync (VSYNC).
 
[[File:pcbSchematic.png|500px|thumb|left|PCB Schematic]]
[[File:pcbLayout.png|500px|thumb|right|PCB Layout]]
[[File:pcbPic-min.png|500px|thumb|right|Manufactured PCB]]
[[File:pcbManufactured.png|500px|thumb|left|Soldered PCB]]
 

=== Hardware Interface ===

Two Digi XBee devices were used to communicate between the SJ2-Board controller and the LPC4088 board to control the paddle on the game. The XBee devices were programmed using the XTCU programming application provided by Digi International. The SJ2-Board used one XBee as a transmitted and the LPC4088 used another as a receiver. The two modules communicated using UART frames and a single byte represented any change of movement or button presses sent by the controller.

{| class="wikitable"
|-
! scope="col"| XCTU
! scope="col"| Transmitter
! scope="col"| Receiver
|-
! scope="row"| CH
| C
| C
|-
! scope="row"| ID
| 6969
| 6969
|-
! scope="row"| CE
| Coordinator
| Endpoint
|-
! scope="row"| Baud Rate
| 9600
| 9600
|-
|}
[[File:sj2Board.png|380px|thumb|center|SJ2 Board]]
[[File:CMPE146_F16_MW_XbeeModule.jpg|200px|thumb|center|XBee Device]]
 

Two LED buttons were used to interface with the controller as inputs. They were both set up using GPIO.
[[File:VGA LED Button.jpg|200px|thumb|center|LED Button]]
 
[[File:VGA_Controller_Block_Diagram.PNG|380px|thumb|center|Controller Block Diagram]]
 

==== I2S Peripheral and Stereo Audio Decoder ====

The LPC4088 is equipped with an I2S stereo encoder. This module provides a hardware-enabled solution for sending I2S-encoded audio data packets to stereo decoders, which produce the corresponding audio waveform. For this project, since 16-bit stereo audio playback was chosen, data packets transmitted on this interface were sent in the following format:

[[File:I2S_audio_data_word.png]]

Here, each word of data encodes a single packet of audio in two 16-bit sections: one signal level for the left channel, and another for the right. These 4 bytes are sent at a rate of 44100 Hz, resulting in a total data transmission of 176400 bytes a second. Rather than needing to be software-defined, this transmission is automatically handled by hardware using a 8-word deep FIFO queue that manages streaming out the audio data at a fixed rate, saving a significant amount of CPU cycles.

This encoded audio signal was sent to a dedicated external [https://www.nxp.com/docs/en/data-sheet/UDA1380.pdf stereo decoder], which output to a 3.5 mm audio jack connected to speakers in the VGA monitor.

==== LCD Peripheral ====

The LCD peripheral was configured for 16-bit color palette (5 bits of color data for each color level -- red, green, and blue -- and an unused intensity bit), at a resolution of 800 by 600 pixels, and with an active color palette size of 256 colors. Since the logic for driving an LCD display has the same concepts (horizontal blanking, vertical blanking, etc) and timing information for any given resolution was the same, we were able to directly drive a VGA display with the same signals and timing data (after the digital signal had been converted to analog).

[[File:LCD_%2B_VGA_Video_signal.png]]

The timing register configurations needed to achieve the requested video display was derived from a dedicated website for [http://tinyvga.com/vga-timing/800x600@60Hz VGA signal info].

==== External Memory ====

The game's visual assets and tracker audio file (both after being loaded from an SD card) and the game's video buffer required more memory than what is present natively in the microcontroller. To accommodate this, an external 32MiB memory module was used. This module was interfaced with using the external memory controller peripheral present on the board.

=== Software Design ===
==== Controller ====

There is an internal accelerometer on the SJ2 board that is utilized as the movement controls for the paddle movement. The y-axis value of the magnitude is polled 17 times in 17 milliseconds and is divided by 17 to get an average reading per that time frame. The maximum tilt value that can be read by the controller is set to 700 out of the total 1000 (highest possible value). The current tilt value is divided by 31 to get a total of 32 possible readings to represent 5 bits of magnitude.

The controller sends one byte of data to the console whenever there is any change in magnitude on the accelerometer or button changes. The byte contains all the information needed to control the paddle on the screen. The first two bits on the byte represent 0 if there is no button input and 1 if there is a button input. The third bit represents whether the controller is tilted left or right with 0 being left and 1 being right. The last five bits represent the controller accelerometer's magnitude.

[[File:VGA_UART_Frame.PNG]]

==== Audio ====

Sound design was provided to the project (in the form of music and sound effects) through the use of a [https://https://en.wikipedia.org/wiki/Music_tracker tracker audio library]. Source code was adopted from the [https://github.com/schismtracker/schismtracker Schism Tracker] library and altered with the following modifications:

- All UI logic was stripped from the source code

- All file handling logic was reworked to use the SD card reading functions, as the project does not support POSIX file calls

- The [https://www.libsdl.org/ Simple DirectMedia Layer] hooks were replaced with a FreeRTOS task that periodically called the waveform generation callback function, which was passed to the I2S peripheral hooked up to a stereo decoder.

=== Implementation ===

==== Ball Logic ====

The ball is controlled by the buttons and the controller accelerometer values. When the button is pressed, the ball is initially sent with a random velocity. The velocity is determined by a substate timer and is initially set as one of 6 possible values.

{| class="wikitable"
|-
! scope="col"| Substate Count
! scope="col"| X-Velocity
! scope="col"| Y-Velocity
|-
! scope="row"| 0
| 4
| -2
|-
! scope="row"| 1
| -4
| -2
|-
! scope="row"| 2
| 2
| -4
|-
! scope="row"| 3
| -2
| -4
|-
! scope="row"| 4
| 1
| -5
|-
! scope="row"| 5
| -1
| -5
|-
|}

The ball's velocity changes whenever it detects a collision with either the side of the screen, the bricks on the top of the level, or the paddle at the bottom. If colliding with the paddle, the ball's velocity will take into account the paddle's velocity and add it to its current velocity. This is done by adding the paddle's x-velocity and adjusting the y-velocity by normalizing the velocity vectors to have a magnitude of 5. This means the ball will always be moving with a magnitude of 5 in any direction after it hits the paddle. If the ball collides with anything other than the paddle, then the velocity of the ball is changed with no increase or decrease of magnitude. If the ball collides on the side of a wall or block, the x-velocity will reflect (go from negative to positive or vice-versa) and the y-velocity will remain constant. If the ball collides on the top or bottom of a block, the y-velocity will reflect and the x-velocity will remain constant. If it appears that the ball collides with the side and top or bottom of something, then the ball will check whichever face it will collide with first and then calculate the other face. Lastly, when the ball collides with the bottom of the screen, then the player experiences a "death" and the ball will reset on the paddle. If the ball collides with the top of the screen, the screen will scroll up for the user to play on the next level.

Whenever the ball collides with a brick, the brick will disappear. To accomplish this, there is a 2-D array of unsigned integers that contains the values of the bricks. If the integer is 0, then there is no block. If the integer is any value other than 0, then the block is filled and has collision logic. If the ball approaches a region that might have a block, then there will be a check within that 2-D to see if the block is filled or empty. If the block is filled, there will be the collision logic that changes the vector of the ball and sets the integer value in the 2-D array to 0 to represent that the block has been broken.

==== Paddle Movement ====
The paddle movement is controlled by the accelerometer data being wirelessly transmitted from the SJ-Two board to the LPC4088 board. The accelerometer data will map to 63 possible indexes for the paddle location. Originally, the paddle would jump directly into the corresponding index location depending on the accelerometer input. We later iterated on this process by implementing an algorithm to smoothly transition into these indexes. This was done by calculating a paddle velocity using the "current desired paddle position" and the previous paddle position. We adjusted the parameters of the algorithm iteratively to produce the smooth paddle movement.

== Testing & Technical Challenges ==

Describe the challenges of your project. What advise would you give yourself or someone else if your project can be started from scratch again?
Make a smooth transition to testing section and described what it took to test your project.

Include sub-sections that list out a problem and solution, such as:

=== VGA Interfacing ===
A DAC resistor ladder was designed and a corresponding custom PCB was acquired to interface the 3.3V digital LCD controller signal with the analog VGA input signal. The VGA cable takes between 0V (completely dark) and 0.7V (maximum brightness) for color pin inputs. The DAC resistor ladder was designed and implemented on a breadboard to verify the design. Validating the design on a breadboard gave us a clear vision during PCB design and planning.

=== VGA Write Timings ===
There were initial tests that were required to be done to see if using a VGA monitor as an output was a viable option for this project. We had to test memory write timings to see if they were fast enough to match the refresh rate of the monitor. Those tests would decide how much of the screen buffer we could write to during the blanking periods during the VGA refresh. This was a good learning experience for testing hardware capabilities and should be done whenever introducing any new aspects to a project.

=== Ball Collision Logic ===
The initial version of the ball collision used pixel values to see if the ball was entering the brick. The problem was that this logic did not account for the case where the ball enters the brick diagonally and the logic for that version of the collision had to be scrapped. The later iteration of the logic was changed to checking whether the ball was potentially crossing into an area that had a brick. This was done by separating the checks into a grid that represented the locations of the blocks. In this case, there was an initial check to see if the ball was crossing a boundary where there might be a new brick. If the ball is crossing the x-boundary and not the y-boundary, then there would be a collision check to see if it collided with a block. If it is crossing the y-boundary and not the x-boundary, once again then there would be a collision check to see if it collided with a block. If it is crossing both the x-boundary and y-boundary, then there would be a check to see which boundary the ball would cross first. This new logic allowed for us to account for the case where the block enters the block diagonally. This taught us that sometimes it is beneficial to scrap a previous iteration and use a new approach to solving a problem because it may lead to better results.

=== Paddle Movement ===
One issue we discovered was the paddle would slow down as it approached the walls. This was due to the difference in "current desired paddle position" and the previous paddle position becoming smaller as the walls were approached, leading to a smaller paddle velocity. To account for this, whenever the accelerometer sent data that mapped to the minimum or maximum paddle index, we would increase the distance of the "current desired paddle position" so the paddle velocity would not slow the paddle down as it approached the walls.

=== Paddle Collision Logic ===
During final integration and testing, one of the biggest issues we encountered was a bug where the ball would not detect a collision with the paddle. The ball would phase right through the paddle instead of bouncing off of it. One of the great ideas that we thought of was to change the color of the ball during each check to see if the checks would go through. This debug check allowed us to verify that section of code that checked for paddle collisions was being skipped even though the ball's next Y position had crossed the paddle's Y position. By having this debugging method, we were able to root cause the issue.

== Conclusion ==
Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

=== Project Video ===
[https://www.youtube.com/watch?v=ipngAGlN86c SJSU CMPE 244 FALL 2021: Embedded Software Project Gameplay]

=== Project Source Code ===
* [https://gitlab.com/fayek.wahhab/sjtwo-c/-/tree/feature/i2s_dma2 Treasure Dive Baseline for Demo Source Code]
* [https://gitlab.com/hobriandh/sjtwo-c/-/tree/final_controller Controller for Demo Source Code]

== References ==
=== Acknowledgement ===
We would like to acknowledge and thank our instructor Preetpal Kang for sharing his knowledge in embedded software development using FreeRTOS.

=== Appendix ===
*[https://gitlab.com/sjtwo-c-dev/sjtwo-c/-/blob/master/LPC408X_7X%20User%20Manual.pdf LPC408x/407x Datasheet]
*[https://www.freertos.org/Documentation/RTOS_book.html FreeRTOS Documentation]
*[http://elm-chan.org/fsw/ff/00index_e.html FatFs - Generic FAT Filesystem Module]
*[https://www.digi.com/resources/library/data-sheets/ds_xbeemultipointmodules Xbee Datasheet]
*[http://books.socialledge.com/books/embedded-drivers-real-time-operating-systems Embedded Drivers & Real Time Operating Systems Lessons]