F22: DDRTOS
Contents
Abstract
Rhythm games have been popular for years and date back to the 1970’s, but hit its stride in 1997 when one of the most notable and popular games released - Dance Dance Revolution (DDR). The main point of a rhythm game is to use music and challenge a player’s ability to stay on beat and typically press a button to the beat of the song. Our project is based and dedicated to DDR, as we all share a passion for music and video games. DDR is played with four arrows for each direction up, down, left, and right that need to be hit at the time of a beat in order to score and increase the multiplier. There are several aspects that are taken from other games like Guitar Hero and Friday Night Funkin’ (sound effects, multiplier, and artwork) that have also become a part of the project. Our project name, DDRTOS, is a combination of the game DDR and RTOS which runs the project.
Objectives & Introduction
Objective
- Create a DDR-like game on RTOS based system, hence the name DDRTOS
- Responsive button press
- Good Audio
- Good visual and audible response when hitting at the right time
Game Introduction
- After the title sequence, you'll be prompted to press any key to start.
- After you press any key, you'll be brought to the song selection screen.
- Press up or down to select song.
- Press right to confirm you choice, and the game starts.
- The arrows are mapped to the beat, so you should aim to hit the button when the static red arrow is fully lit up.
Team Members & Responsibilities
- Michael Hatzikokolakis
- Core gameplay development
- Beatmap development
- Kyle Kwong
- MP3 module driver development
- Audio system research and design
- Ensure coding standards
- Assist on all parts of the project
- Anthony Zunino
- Enclosure Design
- PCB Design
- Converted the mp3 module to use UART queue instead of polling
Schedule
| Week# | Start Date | End Date | Task | Status |
|---|---|---|---|---|
| 1 |
|
|
|
|
| 2 |
|
|
|
|
| 3 |
|
|
|
|
| 4 |
|
|
|
|
| 5 |
|
|
|
|
| 6 |
|
|
|
|
| 7 |
|
|
|
|
| 8 |
|
|
|
|
| 9 |
|
|
|
|
| 10 |
|
|
|
|
Parts List & Cost
Note: For the keycaps, we suggest using uniform profile keycaps such as DSA or XDA.
| Item# | Part Desciption | Vendor | Qty | Cost/Item |
|---|---|---|---|---|
| 1 | 64x64 RGB LED Matrix - 3mm Pitch (192mm x 192mm) | Adafruit [1] | 1 | $59.95 |
| 2 | MP3 Module | Amazon [2] | 1 | $12.99 |
| 3 | PCB | JLCPCB [3] | 1 | $ |
| 4 | NeoKey Socket Breakout for Mechanical Key Switches with NeoPixel – For MX Compatible Switches | Adafruit [4] | 4 | $1.75 |
| 5 | LM7805 Power Regulator | Digi-Key [5] | 1 | $0.75 |
| 6 | 10-pin Terminal Block | Digi-Key [6] | 1 | $3.86 |
| 7 | LEDs | Adafruit [7] | 8 | $5.00 |
| 8 | Mx style switches | Prevail [8] | 4 | $20.00 |
| 9 | Mx style keycaps | Adafruit [9] | 4 | $4.95 |
| 10 | SeeedStudio 4-pin connectors | SeedStudio [10] | 2 | $1.70 |
| 11 | 4-pin to wire connector cable | SeedStudio [11] | 2 | $3.20 |
| 12 | 20-pin ribbon cable | Adafruit [12] | 1 | $1.95 |
| 13 | Miscellaneous resistors and capacitors | N/A | N/A | N/A |
Design & Implementation
The design section can go over your hardware and software design. Organize this section using sub-sections that go over your design and implementation.
How to Reproduce
- PCB
Creation of the PCBs were the highest priority at the start of the project in order to begin development and integration with the entire system. The original idea was to create a single master PCB, but evolved into two boards; one board will be used as the PSU and sound wiring for the SJ2 and LED Matrix, and the other board as a controller for the game both of which including LEDs indicating power, descriptive silkscreen, and easter eggs for DDR.
Original brainstorm idea for PCBs and Enclosure and Timeline
Schematic for the PSU
Schematic for the controller - note that there is a minor error with the switch inputs tied directly to GND and the caps messed up the LEDs
PSU, sound, and LED Matrix power PCB containing two MP3 decoders for multiple sound effects which port to an AUX output, extra 5V/GND pins, a power jack and 5V voltage regulator, and a neat DDR design.
The controller PCB which takes in 5V and GND, and outputs four GPIO switch inputs using SeeedStudio 4-pin connectors.
- 3D Printed Enclosure
Originally, the enclosure’s design was meant to replicate that of an old stand-up retro video game machine. As the design progressed, we wanted to go with a smaller version of the retro machine that was angled and provided both access to the PSU and SJ2 boards in the back while also maintaining a small form factor.
Some original drawing ideas for the base of the enclosure, the enclosure drawing can be seen on the left, and backplate cover in the middle. What is not shown are the pages of mistakes, bad drawings, and measurements down to the 0.01mm for 3D printing.
The enclosure, with a slot for a piece of plastic to slide through and keep the LED Matrix from falling out. The opening in the back allows access to the power and ribbon-cable connections to the matrix
The base which the enclosure connects to with the little tab on the slanted holder. The tabs in the back allow the PSU and SJ2 cases to connect and not jostle around
The PSU, SJ2, and Controller cases from top-left to top-right to bottom. The openings allow access to the terminal block, power jack, micro-USB on SJ2, and the connectors on the controller.
The backplate cover for the base. No connections except some material to make hot-gluing to the base easier.
In order to reproduce the enclosure, use the attached FreeCAD file in ddrtos/enclosure_cad to create .stl files for each part, or use the .stls saved in the ddrtos GitLab repo. Then use a slicer like Cura to generate G-code to upload to a 3D printer.
In order to reproduce the PCBs, take the gerber files located in the ddrtos/eagle_projects/controller and ddrtos/eagle_projects/PSU_Matrix_Sound_Board directories and go to pcbway.com to quote the boards. They take about 10 days (including weekends) to come in.
- Mechanical Keyboard Switches, Keycaps, and Breakout Board
Mechanical switches usually have one to three plastic pins for stabilizing on the PCB. However, those pins also made it would be nearly impossible to use them on a breadboard during early development. As a result, we used the Neokey breakout boards. The breakouts were solely used for the alignment of the yellow switch as seen below:
Top and bottom view of the yellow switches used as switch inputs for the game
The two pins on the bottom side and stabilizing pin made making a PCB footprint for the switch difficult, so a breakout board with standard 0.1” spacing was much easier, just a little more expensive. There is an open source library for MX style footprint for KiCad[13], however, because Eagle does not have support for library conversion, we were not able to take advantage of the library, and as a result, decided to use the breakout board for the PCB design.
As for keycaps, we chose DSA keycaps because Kyle thinks that for a flat controller design, uniform profile keycaps would provide the most comfort.
- Power
- Power bricks come in many different wattages and ratings, so a power jack and voltage regulator were the easiest solution to accommodate different power bricks while still providing a stable voltage for both the LED Matrix, SJ2 board, and controller. The traces on each of the PCBs were widened as much as possible to allow as much current to flow as needed, and the wires and terminal block connections were selected based on the specifications of the Matrix as it was the biggest power draw.
Hardware Interface
In this section, you can describe how your hardware communicates, such as which BUSes used. You can discuss your driver implementation here, such that the Software Design section is isolated to talk about high level workings rather than inner working of your project.
- DFmini MP3 Module
- The MP3 module uses UART to receive instructions from SJ2, with a baud rate of 9600.
- You can select the playback source by sending it the select playback device instruction (0x09). We chose SD card (parameter 0x0002) as our playback source because it requires the least amount of signal processing.
- The module is able to play from 0001.mp3 up to 2999.mp3.
Hardware Design
- PCB
Eagle was used to design the PCBs and PCBWay to order, manufacture, and ship the PCBs to the US. The schematics shown above helped create the bill of material (BOM) to order parts from DigiKey, Adafruit, SparkFun, and Amazon. Once everything came in, it was a matter of soldering everything together and continuity testing to make sure everything would power up without shorting. The scariest part was the power regulator getting hot really quickly after only 5 minutes of minimal LED Matrix display, but otherwise on integration day everything was put together and worked for the most part. The button switches used pull-up resistors to make a rising-edge trigger, but it took a while to realize that the on-board 10k pull-ups on the SJ2 needed to be enabled to properly see the pulse.
A look at the PCBs (almost) fully built, as parts took a lot longer than expected to arrive. Note that the left-most connector on the controller PCB are all connected to GND, but the two yellow LEDs provide two extra GPIOs to provide any information to the player - originally designed to indicate player 1 and 2.
- 3D Printing
Use of 3D printing would help with creating a customizable enclosure and allow for clean connections and precise modeling. FreeCAD is a free, open-source software that is similar to Autodesk CAD and can create .stl files. Then a slicer, Cura, can take the .stl to create a gcode file to upload to a 3D printer. The 3D printer used in the project is the Artillery Sidewinder x1.
The main draw was being able to create custom cases for the PCBs and lock them onto the enclosure. Eventually, it became a hassle to reprint a design for a minor tweak (for example, the plastic store only went down to ⅛” thickness pieces, so the entire enclosure needed to be reprinted just a day before the demo!), so a broader approach like using cheap material would have been a better and quicker solution.
There are a total of six different printed pieces: the enclosure, the base, the PSU case, the controller case, the SJ2 case, and the base backplate. Each has its own FreeCAD design, .stl, and generated gcode. Overall, a total of 84 hours of printing and almost 1kg of PLA filament was used to create four different versions of the base, three different versions of the enclosure, and a single version of each of the cases.
View of the first draft of the enclosure mid-print - 12 hours done (for 1.5mm of height), only 5 more hours to go!
View of the first enclosure draft which takes into account the power/ribbon cable on the back of the LED Matrix, a little bit of tape, and a large breadboard for prototyping to the left
View of the penultimate base print showing the 0.01mm precise fittings for the PCBs. Unfortunately, the cables out of the PSU (on the right) interfered with the enclosure so a final draft was needed for clearance… another 16 hour print!
Full front view of the entire project with the plastic screen, cases, speakers, and wiring.
Full back view with aux cord and a missing power cable to the PSU PCB (on the left) and all the wiring hidden by the backplate base cover.
Software Design
Show your software design. For example, if you are designing an MP3 Player, show the tasks that you are using, and what they are doing at a high level. Do not show the details of the code. For example, do not show exact code, but you may show psuedocode and fragments of code. Keep in mind that you are showing DESIGN of your software, not the inner workings of it.
.png)
.png)
Implementation
This section includes implementation, but again, not the details, just the high level. For example, you can list the steps it takes to communicate over a sensor, or the steps needed to write a page of memory onto SPI Flash. You can include sub-sections for each of your component implementation.
MP3 module driver
- The driver has 5 public functions.
- We dedicated one pin for each UART port
-
void mp3_module__init(uart_e uart_to_use);- This function sets the dedicated pin for the selected UART port to UART mode, initializes the UART, and selects SD card as the playback source. If the GENUINE macro is set to 1, it also sets the volume to 10.
-
void mp3_module__play(uart_e uart_to_use, uint32_t song_number);- This function sends the instruction to the mp3 module to play the song with the name song_number.mp3. It checks to make sure that the song_number is less than or equal to 2999.
-
void mp3_module__resume(uart_e uart_to_use);- This function resumes a paused song.
-
void mp3_module__pause(uart_e uart_to_use);- This function pauses a playing song.
-
void mp3_module__reset_playback(uart_e uart_to_use);- This function resets the module.
| UART Port | UART_0 | UART_1 | UART_2 | UART_3 |
|---|---|---|---|---|
| GPIO Port.Pin | 0.0 | 2.0 | 0.10 | 4.28 |
Title Sequence
// title_sequence.h #pragma once #include "FreeRTOS.h" #include "task.h" TaskHandle_t title_logo_sequence; void title_logo(void *p); void press_to_start__handler(void);
- Overview
- The title sequence serves two purposes. It provides an introductory credits screen, followed by a brief animation of the letters “DDR” in red moving downwards and the letters “RTOS” in blue moving upwards to combine and form “DDRTOS” with the shared letter “R” changed to the color purple. This entire scene is set to the song “Ghost Voices” by Virtual Self, with the names in the credits appearing and disappearing to the beat of the song, and the acronyms merging at the drop of the song. After the acronym merge and a one second delay, the text “Press Anywhere To Start” appears below the logo and the player is able to press any button on the controller to clear the screen, stop the music, and advance to the song select sequence. There is no way to return to the title screen without resetting the system.
- Tasks
- Title Sequence only involves one task, “title_logo”, which steps through each part of the sequence before calling the next sequence and suspending itself.
- Exposed Functions
- The only exposed function is “press_to_start__handler” which is meant to be called from the button.c state machine.
Testing & Technical Challenges
Bugs that had to be worked around
- Audio sync issue
- Even finding standard 120 beat per minute (BPM) songs to set the timing between notes to 500ms did not work. There was always an inherent delay that was not fixed using vTaskDelayUntil()
- We tried enabling the trace library, but it seems that the FreeRTOS version is incompatible with trace library, so we were unable to verify the timing issues with the trace library
- Eventually the beat maps, or the mapping of arrows and timing of when they appear, had to be completely manually made
- Some helpers were created to make the process a little faster, but some delays and a ton of back and forth between compiling and verifying the timing was needed
- Hardware issue with the mp3 module
- Lesson Learned: Don’t cheap out. Always buy genuine products from reputable sites. We wasted many hours because we didn’t buy the genuine product
- Incompatible Instruction format
- The genuine product has a 10-byte instruction format with the second and third last two bytes being the checksum.
- However, the one we purchased didn’t have the checksum implemented, and the product page didn’t mention it. So after we wrote the mp3 module driver, we had no idea why it didn’t work. After spending hours reading the datasheets, doing printfs to the terminal, and even reading the logic analyzer output to check if the SJ2 was sending the correct data to the mp3 module.
- Then we decided to read the reviews and see if others had the same issue, and we found out that the product we bought was not a genuine product, and did not implement all the functionalities that the genuine product has.
- From our testing, the one we have omitted the checksum from the instruction format. Also, volume related instructions are also omitted from the mp3 module.
- However, because the module does not have a built-in DSP, we are unable to make the sound effects overlap.
- Indexing Issue
- Visible scan rate issue
- Unstable Soldering Connection
- The Eagle library for the LED has extremely small pads. As a result of that, we are unable to make good connections on the power indicator LED.
Bugs that got fixed
- Able to move arrows after selecting song, changes song you picked
- Song end timings
- Songs starting at different times due to MP3 module
Hardware limitation and possible solution
Since our mp3 module can only play one file at a time, and does not have a built-in DSP, we used two mp3 modules, one for music playback and the other for sound effects. One of the line signals from the music module connects to the left channel of the aux port, and one of the line signals from the sound effect module connects to the right channel. We didn’t mix the signals into a full stereo because of time constraints. In theory, we can get this done with only one mp3 module by using the onboard DSP on the SJ2 to decode the mp3 files, and sum the song and the sound effect together to create one signal that can be converted into audio with a DAC.
Conclusion
Overall, the project was all encompassing of the topics learned over the semester; from binary semaphores and queues to interrupts to quick handlers, each and every lecture contributed to different parts of the project. There were difficulties in implementing all the tasks in a structured way in the beginning, but, as more code was modularized, a new structure to the codebase was developed to integrate each part of the project - for example, a sound effect could play from the mp3 module which is defined in the drivers layer, and it could be called quickly using a helper function defined in the top game logic layer. This helper is in an interrupt handler that was called by an ISR from a button rising-edge interrupt defined in the buttons module. Having this level of modularity made future implementation and additions quick and well-defined for readability. If things could have been done differently, ordering of all the parts and a very rough idea for how the entire game would progress would have saved a lot of time spent waiting for parts and with initial implementation. If there were more time to work on this project, more work could be added to the state machine to flesh out the game, as a lot of design went into the state machine near the demo deadline. Then, additional features like a high score system are easier to add and create a much more fun and competitive game. Overall, the project helped solidify the concepts learned in CMPE244 and development of the game was much more interesting since the solutions were all based on the background solidified in the first section of the course. There is no limit to what FreeRTOS can do besides the stack, CPU, and mental utilization!
Project Video
Project Source Code
References
Acknowledgement
Any acknowledgement that you may wish to provide can be included here.
References Used
Appendix
You can list the references you used.
