Difference between revisions of "S18: Hand gesture controlled multiplayer game"

From Embedded Systems Learning Academy
Jump to: navigation, search
(16x32 RGB LED Matrix)
(RGB LED Matrix)
Line 302: Line 302:
  
 
=== RGB LED Matrix ===
 
=== RGB LED Matrix ===
These 16x32 5V,2A supply RGB LED Matrix panels require 12 digital pins (6 bit data, 6 bit control). They have 512 bright RGB LEDs arranged in a 16x32 grid on the front. These displays are 'chainable' - connect output to the next input. This display requires about 800 bytes of RAM to buffer the 12-bit color image. As the matrix panel has a 6-bit parallel interface, SPI hardware won't help and hence simple GPIO interface has been used to communicate with the RGB LED Matrix. As these displays don't have built-in PWM control of any kind, the driver is supposed to redraw the screen over and over to 'manually' PWM the whole thing.
+
These 16x32 5V,2A supply RGB LED Matrix panels require 12 digital pins (6 bit data, 6 bit control). They have 512 bright RGB LEDs arranged in a 16x32 grid on the front. These displays are 'chainable' - connect output to the next input. This display requires about 800 bytes of RAM to buffer the 12-bit color image. As the matrix panel has a 6-bit parallel interface, SPI hardware won't help and hence simple GPIO interface has been used to communicate with the RGB LED Matrix. As these displays don't have built-in PWM control of any kind, the driver is supposed to redraw the screen over and over to 'manually' PWM the whole thing.  
 +
 
 +
Pins R1, G1 and B1 deliver data to the top half of the display whereas R2, G2 and B2 deliver data to the bottom half of the display. Pins A, B, C and D select which two rows of the display are currently lit. The LAT signal acts as the strobe and marks the end of a row of data. The CLK signal marks the arrival of each bit of data. OE (output enable) is an active low signal that switches the LEDs off when transitioning from one row to the next.
 +
 
 +
 
  
 
==== Hand Gesture Control Logic ====
 
==== Hand Gesture Control Logic ====

Revision as of 00:12, 17 May 2018

Grading Criteria

  • How well is Software & Hardware Design described?
  • How well can this report be used to reproduce this project?
  • Code Quality
  • Overall Report Quality:
    • Software Block Diagrams
    • Hardware Block Diagrams
      Schematic Quality
    • Quality of technical challenges and solutions adopted.

Project Title

Hand gesture controlled multiplayer game

Abstract

This project involves implementation of a wireless and hand-gesture controlled multiplayer ping pong (2D) game using the Adafruit's 16x32 RGB LED Matrix. It incorporates the on-board RF Nordic wireless transceiver for wireless communication and the on-board acceleration sensor to translate the hand movement into the slider movement on the display.

Objectives & Introduction

Show list of your objectives. This section includes the high level details of your project. You can write about the various sensors or peripherals you used to get your project completed.

Team Members & Responsibilities

  • Akil Khan
    • Design and development of the RGB Led Matrix driver
    • Game logic development, implementation, QA and testing.
  • Abhilash Tuse
    • Game logic development
    • Logic Development for communication between hand gesture control board and controller board
  • Disha Patil
    • Design of PCB using Eagle for Power Supply
    • Hand gesture control logic development using accelerometer
  • Omkar Kale
    • Hand gesture control logic development using accelerometer
    • Logic Development for communication between hand gesture control board and controller board
  • Vishal Shrivastava
    • Design and development of the RGB Led Matrix driver
    • Design of PCB using Eagle for Power Supply

Schedule

Show a simple table or figures that show your scheduled as planned before you started working on the project. Then in another table column, write down the actual schedule so that readers can see the planned vs. actual goals. The point of the schedule is for readers to assess how to pace themselves if they are doing a similar project.

Week# Date Task Status Actual Completion Date
1 04/10/2018
  • Submission of Project Proposals
  • Research on project requirements.
  • Order components and distribute project modules.
  • Completed
  • Completed
  • Completed
  • 03/24/2018
  • 04/10/2018
  • 04/10/2018
2 04/17/2018
  • Developing logic for hand gesture control using accelerometer sensor.
  • Power Supply circuit design and simulation using Multisim
  • Write basic LED display driver to blink individual pixels and set of pixels.
  • Establishing basic wireless communication between 2 SJOne boards.
  • Completed
  • Completed
  • Completed
  • Completed
  • 04/15/2018
  • 04/16/2018
  • 04/17/2018
  • 04/18/2018
3 04/24/2018
  • Develop advanced API's on top of LED display driver: Draw game objects such as sliders and ball.
  • PCB layout design using Eagle and finalizing the schematic.
  • Establish communication between the hand-gesture controller and display node using wireless mesh module.
  • Project report update on the wiki.
  • Completed
  • Completed
  • Completed
  • 05/01/2018
  • 04/29/2018
  • 04/24/2018
4 05/01/2018
  • Developing logic for the ball movement and translating hand gesture control into slider movement.
  • Soldering components and hardware testing on PCB
  • Packaging of hardware board and related components.
  • Project report update on the wiki.
  • Completed
  • Completed
  • 05/01/2018
  • 05/09/2018
5 05/08/2018
  • Integrate ball movement and slider movement logic into display node algorithm.
  • Project report update on the wiki.
  • Completed
  • 05/05/2018
6 04/15/2018
  • Final testing and bug fixes.
  • Complete wiki report and final demo.

Parts List & Cost

Item# Part Manufacturer Part number Quantity Cost($)
1 SJ One Board Preet 3 80.00
2 Adafruit RGB LED Matrix LED Matrix 1 35.00
3 Power Adapter Power Supply 1 7.95
4 Power Bank 800mah Aibocn 2 11.95
5 PCB Board
6 Hardware Excess Solution Barrel Jack, Jumper wires, Connectors 5.00
  • Total Cost:306.955$

Design & Implementation

The design section can go over your hardware and software design. Organize this section using sub-sections that go over your design and implementation.

Hardware Design

System block diagram

Accelerometer

Accelerometer on board
Accelerometer Detection

The SJOne board has an accelerometer which is interfaced on the I2C2 bus. Accelerometers are electromechanical devices that sense either static or dynamic forces of acceleration. Static forces include gravity, while dynamic forces can include vibrations and movement.The measurements are on 3-axis and these values can be calibrated to find the desired values.

In our project, accelerometer is used for detecting the up-down movement of the wrist. We are using accelerometers on two SJ-One Boards for gesture recognition. The movement from accelerometer sensor will be used to control the slider movement on the RGB LED Matrix.

Wireless Module

SJone wireless remote

Nordic wireless chip nRF24L01+ is interfaced with SJone board using SPI bus protocol.

The nRF24L01+ is a single chip 2.4GHz transceiver with an embedded baseband protocol engine, suitable for ultra low power wireless applications. The nRF24L01+ is designed for operation in the frequency band of 2.400 - 2.483GHz. The high air data rate combined with two power saving modes make the nRF24L01+ very suitable for ultra low power designs. The nRF24L01 integrates a complete 2.4GHz RF transceiver, RF synthesizer, and baseband logic including the Enhanced ShockBurst™ hardware protocol accelerator supporting a high-speed SPI interface for the application controller. No external loop filter, resonators, or VCO varactor diodes are required, only a low-cost ±60ppm crystal, matching circuitry, and antenna.

RGB LED Matrix

Pin mapping of RGB LED matrix with SJ One board:

RGB LED matrix Pins SJ One Board Pins Function
R1 P1.19 Red Data top half
G1 P1.20 Green Data top half
B1 P1.22 Blue Data top half
R2 P1.23 Red Data bottom half
G2 P1.28 Green Data bottom half
B2 P1.29 Blue Data bottom half
A P2.0 Row select A
B P2.1 Row select B
C P2.2 Row select C
CLK P0.1 The CLK (clock) signal marks the arrival of each bit of data.
OE P0.0 OE (output enable) switches the LEDs off when transitioning from one row to the next.
LAT P0.26 The LAT (latch) signal marks the end of a row of data.

Hardware Interface Diagram

In this section, you can describe how your hardware communicates, such as which BUSes used. You can discuss your driver implementation here, such that the Software Design section is isolated to talk about high level workings rather than inner working of your project.

Hardware Interface of RGB LED matrix

Software Design

Show your software design. For example, if you are designing an MP3 Player, show the tasks that you are using, and what they are doing at a high level. Do not show the details of the code. For example, do not show exact code, but you may show psuedocode and fragments of code. Keep in mind that you are showing DESIGN of your software, not the inner workings of it.

RGB LED Matrix

These 16x32 5V,2A supply RGB LED Matrix panels require 12 digital pins (6 bit data, 6 bit control). They have 512 bright RGB LEDs arranged in a 16x32 grid on the front. These displays are 'chainable' - connect output to the next input. This display requires about 800 bytes of RAM to buffer the 12-bit color image. As the matrix panel has a 6-bit parallel interface, SPI hardware won't help and hence simple GPIO interface has been used to communicate with the RGB LED Matrix. As these displays don't have built-in PWM control of any kind, the driver is supposed to redraw the screen over and over to 'manually' PWM the whole thing.

Pins R1, G1 and B1 deliver data to the top half of the display whereas R2, G2 and B2 deliver data to the bottom half of the display. Pins A, B, C and D select which two rows of the display are currently lit. The LAT signal acts as the strobe and marks the end of a row of data. The CLK signal marks the arrival of each bit of data. OE (output enable) is an active low signal that switches the LEDs off when transitioning from one row to the next.


Hand Gesture Control Logic

Implementation

This section includes implementation, but again, not the details, just the high level. For example, you can list the steps it takes to communicate over a sensor, or the steps needed to write a page of memory onto SPI Flash. You can include sub-sections for each of your component implementation.

Testing

Module testing

Acceleration Measurement

Hand gesture recognition was rigorously tested. The values received from accelerometer sensor were within a range. So, we had to test the range of values received at which the slider on screen looked responsive and smooth. We tested values ranging from -1000 to +1000 to find the right range of values. The hand gesture module transmits uint8_t. So, we checked whether the output at minimum and maximum input values was within valid range. We also checked the uint8_t value(index) transmitted depending upon the inputs received from the accelerometer sensor.

LCD Display

Integration testing

PCB Testing

Technical Challenges

PCB Designing

Task Synchronization Issue

RGB LED Matrix Refresh Issue

Conclusion

Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

Project Video

Upload a video of your project and post the link here.

Project Source Code

References

Acknowledgement

We would like to thank our Professor Preetpal Kang for all his teachings and inspirational lectures. Not only did we enjoy working though out this project but also gave us an overall learning experience and precious life lessons. We would also like to thank the ISA members for always being ready to help with whatever issues we faced.

References Used

[1] FreeRTOS documentations [2] Adafruit LCD library [3] CMPE 244 Lecture notes from Preetpal Kang, Computer Engineering, San Jose State University. Feb - May 2018. [4] Nordic wireless datasheet