Embedded Systems Learning Academy - User contributions [en]

F16: Object Detector

2016-12-21T18:39:57Z

146 user5: /* Team Members & Responsibilities */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
TEAM 5
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
**Programmed pre-determined objects for Pixy to recognize

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: CmpE146 F16 T5 Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center>]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: CmpE146_F16_T5_Pixy_Camera.jpg| frame | center|<center>'''Figure 2. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File:CmpE146_F16_T5_Pixy_Camera_Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: CmpE146_F16_T5_Servo_Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receiving and sending bytes of information. Below is the Pseudocode an explanation for each function utilized within the UART driver.

<center><big> ''Initialization''</big> </center>

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below. From this equation we derive the DLL and DLM registers which are used as the divisors to establish the desired baud rate for the data transfer. The DLAB bit must also be initialized at is used as both the Receiver Buffer Register and the Transmit Holding Register. Both of which are set at the LCR register. The LCR register is also responsible for setting the start and stop bits of the UART communication.

[[File: CmpE146 F16 T5 BaudRate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

''Initialization PseudoCode''

PCONP = Power on UART2
PCLKSEL1 = Set Peripheral Clock
PINSEL4 = Set RX and TX pin functionalities
// Acquire DLL_Value from desired Baud Rate
DLL_Value = clock/(16*Baud)
Set DLM and DLL registers according to DLL_Value
LCR = Set 8 bit data transfer Length
LCR = Set start and stop bit for data transfer

<center><big> ''Receiving Data''</big> </center>

Since the SJOne board is only receiving data from the Pixy Camera and not sending data in this situation only a receive byte function is required. The function waits to see if the data is transmitted from the Pixy Camera to the SJOne board by checking the RDR (Receiver Data Ready) register. Once the value of this register is set to 1, the receive function can then capture the transmitted data by looking at the RBR (Receiver Buffer Register).

''Receiving Function PseudoCode''

while (1){
if(RDR){
break;
}
}
new_data = RBR;

=== Pixy Camera Software Implementation ===

The Pixy Camera is able to externally communicate with micro-controllers by sending data regarding the object it is tracking, as well as receiving data for controlling the Pixy Camera itself. When the Pixy camera locates a recognized or preprogrammed object it begins to send information to a connected peripheral in bytes regarding that object. The first few bytes the Pixy camera sends are sync bytes, in other words bytes that let the external device know that the Pixy Camera has picked up on an object. The rest of the following bytes indicate the signature of the object, where it is on the screen with respect to the camera view (x/y center), as well as the width and the height of the object. Below are the specific bytes the Pixy Camera send to the SJOne board when it picks up on a device.

[[File: CmpE146 F16 T5 Pixy ObjectBlock.png| frame | center| <center> '''Figure 7. Pixy Camera Object Block Data transfer''' </center>]]

The SJOne board is also able to control information to the Pixy Camera in order to control the pan and tilt of the servos, LED color, and camera brightness. However, the pan and tilt byte information is our area of concern with respect to control information being sent to the Pixy Camera. If the correct bytes are sent then the SJOne board acts as a master driving the pan and tilt servos of the Pixy Camera.

[[File: CmpE146 F16 T5 Servo Control.png| frame | center| <center> '''Figure 8. Pan & Tilt Byte information''' </center>]]

=== Implementation ===

Initially, the Pixy Camera is programmed to recognize a specific object that appears into its view. The UART hardware connections are then made between the Pixy Camera and the SJOne board. The SJOne board registers are initialized and the baud rate is set to match the baud rate of the Pixy Camera. The task then continuously runs the read function in order to continuously grab data from the Pixy Camera. The SJOne board then continuously checks to see if the received data from the Pixy Camera matches the sync byte 0xaa55. This tells the SJOne board that the camera has picked up on an object. Once this byte is received the following bytes will include information regarding the recognized object. Below is the flow of the UART communication, as well as the Pseudo Code behind the implementation.

[[File: CmpE146 F16 T5 UART Flow Chart.png| border|400px|center| <center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>]]

<center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>

''Implementation Pseudo Code''

init(Baud_Rate); // initialize Baud Rate and UART registers
while(1){
data_byte1 = read_byte();
data_byte2 = read_byte();
if (data_byte1 & data_byte2 == 0xaa55){ // 0xaa55 means Pixy has recognized the object and the following 12bytes sent will describe that object.
for (i < 11) // Allows the following 12 bytes of object information to be grabbed
grab_object_data = read_byte();
}
}

Once the data is retrieved from the SJOne board, an application was added to where the SJOne board would be able to identify objects based on the bytes it had received from the Pixy Camera. Utilizing this application the SJOne board was able to recognize a pair of green scissors, an orange, and a blue box.

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion, we were able to complete necessary for our project. All though we done what was needed, we believe that it isn't enough and we could've have done more. When we were testing we ran into problems like every other group, but we able to navigate past them successfully. We are glad we took this Embedded Systems class, because we able to learn how FREE RTOS operating system works and how we can implement it for our project. We have also learned how to implement different types of communication protocols and how to program their drivers. The main part we learned from this project is to make sure to set a team leader and make sure people follow their lead to finish the tasks. Within a group project, you have to hold your teammates accountable, make sure you set deadlines, make sure people are getting their work done and this is pretty important in real work place situations.

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

https://youtu.be/OeIFKckJA0c

=== Project Source Code ===
https://github.com/nsarras/Object_Detector

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.
*http://cmucam.org/projects/cmucam5/wiki/Pixy_Serial_Protocol
*http://cmucam.org/projects/cmucam5/wiki/Porting_Guide
*https://learn.sparkfun.com/tutorials/pulse-width-modulation
*http://www.socialledge.com/sjsu/index.php?title=Realtime_OS_on_Embedded_Systems

F16: Object Detector

2016-12-21T18:15:05Z

146 user5: /* Implementation */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: CmpE146 F16 T5 Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center>]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: CmpE146_F16_T5_Pixy_Camera.jpg| frame | center|<center>'''Figure 2. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File:CmpE146_F16_T5_Pixy_Camera_Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: CmpE146_F16_T5_Servo_Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receiving and sending bytes of information. Below is the Pseudocode an explanation for each function utilized within the UART driver.

<center><big> ''Initialization''</big> </center>

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below. From this equation we derive the DLL and DLM registers which are used as the divisors to establish the desired baud rate for the data transfer. The DLAB bit must also be initialized at is used as both the Receiver Buffer Register and the Transmit Holding Register. Both of which are set at the LCR register. The LCR register is also responsible for setting the start and stop bits of the UART communication.

[[File: CmpE146 F16 T5 BaudRate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

''Initialization PseudoCode''

PCONP = Power on UART2
PCLKSEL1 = Set Peripheral Clock
PINSEL4 = Set RX and TX pin functionalities
// Acquire DLL_Value from desired Baud Rate
DLL_Value = clock/(16*Baud)
Set DLM and DLL registers according to DLL_Value
LCR = Set 8 bit data transfer Length
LCR = Set start and stop bit for data transfer

<center><big> ''Receiving Data''</big> </center>

Since the SJOne board is only receiving data from the Pixy Camera and not sending data in this situation only a receive byte function is required. The function waits to see if the data is transmitted from the Pixy Camera to the SJOne board by checking the RDR (Receiver Data Ready) register. Once the value of this register is set to 1, the receive function can then capture the transmitted data by looking at the RBR (Receiver Buffer Register).

''Receiving Function PseudoCode''

while (1){
if(RDR){
break;
}
}
new_data = RBR;

=== Pixy Camera Software Implementation ===

The Pixy Camera is able to externally communicate with micro-controllers by sending data regarding the object it is tracking, as well as receiving data for controlling the Pixy Camera itself. When the Pixy camera locates a recognized or preprogrammed object it begins to send information to a connected peripheral in bytes regarding that object. The first few bytes the Pixy camera sends are sync bytes, in other words bytes that let the external device know that the Pixy Camera has picked up on an object. The rest of the following bytes indicate the signature of the object, where it is on the screen with respect to the camera view (x/y center), as well as the width and the height of the object. Below are the specific bytes the Pixy Camera send to the SJOne board when it picks up on a device.

[[File: CmpE146 F16 T5 Pixy ObjectBlock.png| frame | center| <center> '''Figure 7. Pixy Camera Object Block Data transfer''' </center>]]

The SJOne board is also able to control information to the Pixy Camera in order to control the pan and tilt of the servos, LED color, and camera brightness. However, the pan and tilt byte information is our area of concern with respect to control information being sent to the Pixy Camera. If the correct bytes are sent then the SJOne board acts as a master driving the pan and tilt servos of the Pixy Camera.

[[File: CmpE146 F16 T5 Servo Control.png| frame | center| <center> '''Figure 8. Pan & Tilt Byte information''' </center>]]

=== Implementation ===

Initially, the Pixy Camera is programmed to recognize a specific object that appears into its view. The UART hardware connections are then made between the Pixy Camera and the SJOne board. The SJOne board registers are initialized and the baud rate is set to match the baud rate of the Pixy Camera. The task then continuously runs the read function in order to continuously grab data from the Pixy Camera. The SJOne board then continuously checks to see if the received data from the Pixy Camera matches the sync byte 0xaa55. This tells the SJOne board that the camera has picked up on an object. Once this byte is received the following bytes will include information regarding the recognized object. Below is the flow of the UART communication, as well as the Pseudo Code behind the implementation.

[[File: CmpE146 F16 T5 UART Flow Chart.png| border|400px|center| <center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>]]

<center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>

''Implementation Pseudo Code''

init(Baud_Rate); // initialize Baud Rate and UART registers
while(1){
data_byte1 = read_byte();
data_byte2 = read_byte();
if (data_byte1 & data_byte2 == 0xaa55){ // 0xaa55 means Pixy has recognized the object and the following 12bytes sent will describe that object.
for (i < 11) // Allows the following 12 bytes of object information to be grabbed
grab_object_data = read_byte();
}
}

Once the data is retrieved from the SJOne board, an application was added to where the SJOne board would be able to identify objects based on the bytes it had received from the Pixy Camera. Utilizing this application the SJOne board was able to recognize a pair of green scissors, an orange, and a blue box.

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion, we were able to complete necessary for our project. All though we done what was needed, we believe that it isn't enough and we could've have done more. When we were testing we ran into problems like every other group, but we able to navigate past them successfully. We are glad we took this Embedded Systems class, because we able to learn how FREE RTOS operating system works and how we can implement it for our project. We have also learned how to implement different types of communication protocols and how to program their drivers. The main part we learned from this project is to make sure to set a team leader and make sure people follow their lead to finish the tasks. Within a group project, you have to hold your teammates accountable, make sure you set deadlines, make sure people are getting their work done and this is pretty important in real work place situations.

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

https://youtu.be/OeIFKckJA0c

=== Project Source Code ===
https://github.com/nsarras/Object_Detector

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.
*http://cmucam.org/projects/cmucam5/wiki/Pixy_Serial_Protocol
*http://cmucam.org/projects/cmucam5/wiki/Porting_Guide
*https://learn.sparkfun.com/tutorials/pulse-width-modulation
*http://www.socialledge.com/sjsu/index.php?title=Realtime_OS_on_Embedded_Systems

F16: Object Detector

2016-12-21T18:10:38Z

146 user5: /* Pixy Camera Software Implementation */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: CmpE146 F16 T5 Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center>]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: CmpE146_F16_T5_Pixy_Camera.jpg| frame | center|<center>'''Figure 2. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File:CmpE146_F16_T5_Pixy_Camera_Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: CmpE146_F16_T5_Servo_Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receiving and sending bytes of information. Below is the Pseudocode an explanation for each function utilized within the UART driver.

<center><big> ''Initialization''</big> </center>

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below. From this equation we derive the DLL and DLM registers which are used as the divisors to establish the desired baud rate for the data transfer. The DLAB bit must also be initialized at is used as both the Receiver Buffer Register and the Transmit Holding Register. Both of which are set at the LCR register. The LCR register is also responsible for setting the start and stop bits of the UART communication.

[[File: CmpE146 F16 T5 BaudRate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

''Initialization PseudoCode''

PCONP = Power on UART2
PCLKSEL1 = Set Peripheral Clock
PINSEL4 = Set RX and TX pin functionalities
// Acquire DLL_Value from desired Baud Rate
DLL_Value = clock/(16*Baud)
Set DLM and DLL registers according to DLL_Value
LCR = Set 8 bit data transfer Length
LCR = Set start and stop bit for data transfer

<center><big> ''Receiving Data''</big> </center>

Since the SJOne board is only receiving data from the Pixy Camera and not sending data in this situation only a receive byte function is required. The function waits to see if the data is transmitted from the Pixy Camera to the SJOne board by checking the RDR (Receiver Data Ready) register. Once the value of this register is set to 1, the receive function can then capture the transmitted data by looking at the RBR (Receiver Buffer Register).

''Receiving Function PseudoCode''

while (1){
if(RDR){
break;
}
}
new_data = RBR;

=== Pixy Camera Software Implementation ===

The Pixy Camera is able to externally communicate with micro-controllers by sending data regarding the object it is tracking, as well as receiving data for controlling the Pixy Camera itself. When the Pixy camera locates a recognized or preprogrammed object it begins to send information to a connected peripheral in bytes regarding that object. The first few bytes the Pixy camera sends are sync bytes, in other words bytes that let the external device know that the Pixy Camera has picked up on an object. The rest of the following bytes indicate the signature of the object, where it is on the screen with respect to the camera view (x/y center), as well as the width and the height of the object. Below are the specific bytes the Pixy Camera send to the SJOne board when it picks up on a device.

[[File: CmpE146 F16 T5 Pixy ObjectBlock.png| frame | center| <center> '''Figure 7. Pixy Camera Object Block Data transfer''' </center>]]

The SJOne board is also able to control information to the Pixy Camera in order to control the pan and tilt of the servos, LED color, and camera brightness. However, the pan and tilt byte information is our area of concern with respect to control information being sent to the Pixy Camera. If the correct bytes are sent then the SJOne board acts as a master driving the pan and tilt servos of the Pixy Camera.

[[File: CmpE146 F16 T5 Servo Control.png| frame | center| <center> '''Figure 8. Pan & Tilt Byte information''' </center>]]

=== Implementation ===

Initially, the Pixy Camera is programmed to recognize a specific object that appears into its view. The UART hardware connections are then made between the Pixy Camera and the SJOne board. The SJOne board registers are initialized and the baud rate is set to match the baud rate of the Pixy Camera. The task then continuously runs the read function in order to continuously grab data from the Pixy Camera. The SJOne board then continuously checks to see if the received data from the Pixy Camera matches the sync byte 0xaa55. This tells the SJOne board that the camera has picked up on an object. Once this byte is received the following bytes will include information regarding the recognized object. Below is the flow of the UART communication, as well as the Pseudo Code behind the implementation.

[[File: Pixy to SJOne board UART flowchart.png| border|400px|center| <center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>]]

<center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>

''Implementation Pseudo Code''

init(Baud_Rate); // initialize Baud Rate and UART registers
while(1){
data_byte1 = read_byte();
data_byte2 = read_byte();
if (data_byte1 & data_byte2 == 0xaa55){ // 0xaa55 means Pixy has recognized the object and the following 12bytes sent will describe that object.
for (i < 11) // Allows the following 12 bytes of object information to be grabbed
grab_object_data = read_byte();
}
}

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion, we were able to complete necessary for our project. All though we done what was needed, we believe that it isn't enough and we could've have done more. When we were testing we ran into problems like every other group, but we able to navigate past them successfully. We are glad we took this Embedded Systems class, because we able to learn how FREE RTOS operating system works and how we can implement it for our project. We have also learned how to implement different types of communication protocols and how to program their drivers. The main part we learned from this project is to make sure to set a team leader and make sure people follow their lead to finish the tasks. Within a group project, you have to hold your teammates accountable, make sure you set deadlines, make sure people are getting their work done and this is pretty important in real work place situations.

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

https://youtu.be/OeIFKckJA0c

=== Project Source Code ===
https://github.com/nsarras/Object_Detector

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.
*http://cmucam.org/projects/cmucam5/wiki/Pixy_Serial_Protocol
*http://cmucam.org/projects/cmucam5/wiki/Porting_Guide
*https://learn.sparkfun.com/tutorials/pulse-width-modulation
*http://www.socialledge.com/sjsu/index.php?title=Realtime_OS_on_Embedded_Systems

File:CmpE146 F16 T5 UART Flow Chart.png

2016-12-21T18:01:06Z

146 user5: UART communication flow chart between Pixy Camera and SJOneboard

UART communication flow chart between Pixy Camera and SJOneboard

File:CmpE146 F16 T5 Servo Control.png

2016-12-21T17:59:57Z

146 user5: Pixy Servo Controls

Pixy Servo Controls

F16: Object Detector

2016-12-21T17:45:21Z

146 user5: /* Pixy Camera Software Implementation */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: CmpE146 F16 T5 Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center>]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: CmpE146_F16_T5_Pixy_Camera.jpg| frame | center|<center>'''Figure 2. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File:CmpE146_F16_T5_Pixy_Camera_Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: CmpE146_F16_T5_Servo_Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receiving and sending bytes of information. Below is the Pseudocode an explanation for each function utilized within the UART driver.

<center><big> ''Initialization''</big> </center>

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below. From this equation we derive the DLL and DLM registers which are used as the divisors to establish the desired baud rate for the data transfer. The DLAB bit must also be initialized at is used as both the Receiver Buffer Register and the Transmit Holding Register. Both of which are set at the LCR register. The LCR register is also responsible for setting the start and stop bits of the UART communication.

[[File: CmpE146 F16 T5 BaudRate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

''Initialization PseudoCode''

PCONP = Power on UART2
PCLKSEL1 = Set Peripheral Clock
PINSEL4 = Set RX and TX pin functionalities
// Acquire DLL_Value from desired Baud Rate
DLL_Value = clock/(16*Baud)
Set DLM and DLL registers according to DLL_Value
LCR = Set 8 bit data transfer Length
LCR = Set start and stop bit for data transfer

<center><big> ''Receiving Data''</big> </center>

Since the SJOne board is only receiving data from the Pixy Camera and not sending data in this situation only a receive byte function is required. The function waits to see if the data is transmitted from the Pixy Camera to the SJOne board by checking the RDR (Receiver Data Ready) register. Once the value of this register is set to 1, the receive function can then capture the transmitted data by looking at the RBR (Receiver Buffer Register).

''Receiving Function PseudoCode''

while (1){
if(RDR){
break;
}
}
new_data = RBR;

=== Pixy Camera Software Implementation ===

The Pixy Camera is able to externally communicate with micro-controllers by sending data regarding the object it is tracking, as well as receiving data for controlling the Pixy Camera itself. When the Pixy camera locates a recognized or preprogrammed object it begins to send information to a connected peripheral in bytes regarding that object. The first few bytes the Pixy camera sends are sync bytes, in other words bytes that let the external device know that the Pixy Camera has picked up on an object. The rest of the following bytes indicate the signature of the object, where it is on the screen with respect to the camera view (x/y center), as well as the width and the height of the object. Below are the specific bytes the Pixy Camera send to the SJOne board when it picks up on a device.

[[File: CmpE146 F16 T5 Pixy ObjectBlock.png| frame | center| <center> '''Figure 7. Pixy Camera Object Block Data transfer''' </center>]]

The SJOne board is also able to control information to the Pixy Camera in order to control the pan and tilt of the servos, LED color, and camera brightness. However, the pan and tilt byte information is our area of concern with respect to control information being sent to the Pixy Camera. If the correct bytes are sent then the SJOne board acts as a master driving the pan and tilt servos of the Pixy Camera.

[[File: Pan & Tilt-Servo Control.png| frame | center| <center> '''Figure 8. Pan & Tilt Byte information''' </center>]]

=== Implementation ===

Initially, the Pixy Camera is programmed to recognize a specific object that appears into its view. The UART hardware connections are then made between the Pixy Camera and the SJOne board. The SJOne board registers are initialized and the baud rate is set to match the baud rate of the Pixy Camera. The task then continuously runs the read function in order to continuously grab data from the Pixy Camera. The SJOne board then continuously checks to see if the received data from the Pixy Camera matches the sync byte 0xaa55. This tells the SJOne board that the camera has picked up on an object. Once this byte is received the following bytes will include information regarding the recognized object. Below is the flow of the UART communication, as well as the Pseudo Code behind the implementation.

[[File: Pixy to SJOne board UART flowchart.png| border|400px|center| <center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>]]

<center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>

''Implementation Pseudo Code''

init(Baud_Rate); // initialize Baud Rate and UART registers
while(1){
data_byte1 = read_byte();
data_byte2 = read_byte();
if (data_byte1 & data_byte2 == 0xaa55){ // 0xaa55 means Pixy has recognized the object and the following 12bytes sent will describe that object.
for (i < 11) // Allows the following 12 bytes of object information to be grabbed
grab_object_data = read_byte();
}
}

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion, we were able to complete necessary for our project. All though we done what was needed, we believe that it isn't enough and we could've have done more. When we were testing we ran into problems like every other group, but we able to navigate past them successfully. We are glad we took this Embedded Systems class, because we able to learn how FREE RTOS operating system works and how we can implement it for our project. We have also learned how to implement different types of communication protocols and how to program their drivers. The main part we learned from this project is to make sure to set a team leader and make sure people follow their lead to finish the tasks. Within a group project, you have to hold your teammates accountable, make sure you set deadlines, make sure people are getting their work done and this is pretty important in real work place situations.

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

https://youtu.be/OeIFKckJA0c

=== Project Source Code ===
https://github.com/nsarras/Object_Detector

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.
*http://cmucam.org/projects/cmucam5/wiki/Pixy_Serial_Protocol
*http://cmucam.org/projects/cmucam5/wiki/Porting_Guide
*https://learn.sparkfun.com/tutorials/pulse-width-modulation
*http://www.socialledge.com/sjsu/index.php?title=Realtime_OS_on_Embedded_Systems

File:CmpE146 F16 T5 Pixy ObjectBlock.png

2016-12-21T17:44:31Z

146 user5: Pixy Object Block output

Pixy Object Block output

F16: Object Detector

2016-12-21T17:42:55Z

146 user5: /* Software Design SJOne Board */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: CmpE146 F16 T5 Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center>]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: CmpE146_F16_T5_Pixy_Camera.jpg| frame | center|<center>'''Figure 2. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File:CmpE146_F16_T5_Pixy_Camera_Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: CmpE146_F16_T5_Servo_Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receiving and sending bytes of information. Below is the Pseudocode an explanation for each function utilized within the UART driver.

<center><big> ''Initialization''</big> </center>

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below. From this equation we derive the DLL and DLM registers which are used as the divisors to establish the desired baud rate for the data transfer. The DLAB bit must also be initialized at is used as both the Receiver Buffer Register and the Transmit Holding Register. Both of which are set at the LCR register. The LCR register is also responsible for setting the start and stop bits of the UART communication.

[[File: CmpE146 F16 T5 BaudRate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

''Initialization PseudoCode''

PCONP = Power on UART2
PCLKSEL1 = Set Peripheral Clock
PINSEL4 = Set RX and TX pin functionalities
// Acquire DLL_Value from desired Baud Rate
DLL_Value = clock/(16*Baud)
Set DLM and DLL registers according to DLL_Value
LCR = Set 8 bit data transfer Length
LCR = Set start and stop bit for data transfer

<center><big> ''Receiving Data''</big> </center>

Since the SJOne board is only receiving data from the Pixy Camera and not sending data in this situation only a receive byte function is required. The function waits to see if the data is transmitted from the Pixy Camera to the SJOne board by checking the RDR (Receiver Data Ready) register. Once the value of this register is set to 1, the receive function can then capture the transmitted data by looking at the RBR (Receiver Buffer Register).

''Receiving Function PseudoCode''

while (1){
if(RDR){
break;
}
}
new_data = RBR;

=== Pixy Camera Software Implementation ===

The Pixy Camera is able to externally communicate with micro-controllers by sending data regarding the object it is tracking, as well as receiving data for controlling the Pixy Camera itself. When the Pixy camera locates a recognized or preprogrammed object it begins to send information to a connected peripheral in bytes regarding that object. The first few bytes the Pixy camera sends are sync bytes, in other words bytes that let the external device know that the Pixy Camera has picked up on an object. The rest of the following bytes indicate the signature of the object, where it is on the screen with respect to the camera view (x/y center), as well as the width and the height of the object. Below are the specific bytes the Pixy Camera send to the SJOne board when it picks up on a device.

[[File: Pixy_ObjectBlock.png| frame | center| <center> '''Figure 7. Pixy Camera Object Block Data transfer''' </center>]]

The SJOne board is also able to control information to the Pixy Camera in order to control the pan and tilt of the servos, LED color, and camera brightness. However, the pan and tilt byte information is our area of concern with respect to control information being sent to the Pixy Camera. If the correct bytes are sent then the SJOne board acts as a master driving the pan and tilt servos of the Pixy Camera.

[[File: Pan & Tilt-Servo Control.png| frame | center| <center> '''Figure 8. Pan & Tilt Byte information''' </center>]]

=== Implementation ===

Initially, the Pixy Camera is programmed to recognize a specific object that appears into its view. The UART hardware connections are then made between the Pixy Camera and the SJOne board. The SJOne board registers are initialized and the baud rate is set to match the baud rate of the Pixy Camera. The task then continuously runs the read function in order to continuously grab data from the Pixy Camera. The SJOne board then continuously checks to see if the received data from the Pixy Camera matches the sync byte 0xaa55. This tells the SJOne board that the camera has picked up on an object. Once this byte is received the following bytes will include information regarding the recognized object. Below is the flow of the UART communication, as well as the Pseudo Code behind the implementation.

[[File: Pixy to SJOne board UART flowchart.png| border|400px|center| <center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>]]

<center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>

''Implementation Pseudo Code''

init(Baud_Rate); // initialize Baud Rate and UART registers
while(1){
data_byte1 = read_byte();
data_byte2 = read_byte();
if (data_byte1 & data_byte2 == 0xaa55){ // 0xaa55 means Pixy has recognized the object and the following 12bytes sent will describe that object.
for (i < 11) // Allows the following 12 bytes of object information to be grabbed
grab_object_data = read_byte();
}
}

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion, we were able to complete necessary for our project. All though we done what was needed, we believe that it isn't enough and we could've have done more. When we were testing we ran into problems like every other group, but we able to navigate past them successfully. We are glad we took this Embedded Systems class, because we able to learn how FREE RTOS operating system works and how we can implement it for our project. We have also learned how to implement different types of communication protocols and how to program their drivers. The main part we learned from this project is to make sure to set a team leader and make sure people follow their lead to finish the tasks. Within a group project, you have to hold your teammates accountable, make sure you set deadlines, make sure people are getting their work done and this is pretty important in real work place situations.

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

https://youtu.be/OeIFKckJA0c

=== Project Source Code ===
https://github.com/nsarras/Object_Detector

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.
*http://cmucam.org/projects/cmucam5/wiki/Pixy_Serial_Protocol
*http://cmucam.org/projects/cmucam5/wiki/Porting_Guide
*https://learn.sparkfun.com/tutorials/pulse-width-modulation
*http://www.socialledge.com/sjsu/index.php?title=Realtime_OS_on_Embedded_Systems

F16: Object Detector

2016-12-21T17:26:28Z

146 user5: /* Project Video */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: CmpE146 F16 T5 Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center>]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: CmpE146_F16_T5_Pixy_Camera.jpg| frame | center|<center>'''Figure 2. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File:CmpE146_F16_T5_Pixy_Camera_Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: CmpE146_F16_T5_Servo_Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receiving and sending bytes of information. Below is the Pseudocode an explanation for each function utilized within the UART driver.

<center><big> ''Initialization''</big> </center>

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below. From this equation we derive the DLL and DLM registers which are used as the divisors to establish the desired baud rate for the data transfer. The DLAB bit must also be initialized at is used as both the Receiver Buffer Register and the Transmit Holding Register. Both of which are set at the LCR register. The LCR register is also responsible for setting the start and stop bits of the UART communication.

[[File: Baud Rate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

''Initialization PseudoCode''

PCONP = Power on UART2
PCLKSEL1 = Set Peripheral Clock
PINSEL4 = Set RX and TX pin functionalities
// Acquire DLL_Value from desired Baud Rate
DLL_Value = clock/(16*Baud)
Set DLM and DLL registers according to DLL_Value
LCR = Set 8 bit data transfer Length
LCR = Set start and stop bit for data transfer

<center><big> ''Receiving Data''</big> </center>

Since the SJOne board is only receiving data from the Pixy Camera and not sending data in this situation only a receive byte function is required. The function waits to see if the data is transmitted from the Pixy Camera to the SJOne board by checking the RDR (Receiver Data Ready) register. Once the value of this register is set to 1, the receive function can then capture the transmitted data by looking at the RBR (Receiver Buffer Register).

''Receiving Function PseudoCode''

while (1){
if(RDR){
break;
}
}
new_data = RBR;

=== Pixy Camera Software Implementation ===

The Pixy Camera is able to externally communicate with micro-controllers by sending data regarding the object it is tracking, as well as receiving data for controlling the Pixy Camera itself. When the Pixy camera locates a recognized or preprogrammed object it begins to send information to a connected peripheral in bytes regarding that object. The first few bytes the Pixy camera sends are sync bytes, in other words bytes that let the external device know that the Pixy Camera has picked up on an object. The rest of the following bytes indicate the signature of the object, where it is on the screen with respect to the camera view (x/y center), as well as the width and the height of the object. Below are the specific bytes the Pixy Camera send to the SJOne board when it picks up on a device.

[[File: Pixy_ObjectBlock.png| frame | center| <center> '''Figure 7. Pixy Camera Object Block Data transfer''' </center>]]

The SJOne board is also able to control information to the Pixy Camera in order to control the pan and tilt of the servos, LED color, and camera brightness. However, the pan and tilt byte information is our area of concern with respect to control information being sent to the Pixy Camera. If the correct bytes are sent then the SJOne board acts as a master driving the pan and tilt servos of the Pixy Camera.

[[File: Pan & Tilt-Servo Control.png| frame | center| <center> '''Figure 8. Pan & Tilt Byte information''' </center>]]

=== Implementation ===

Initially, the Pixy Camera is programmed to recognize a specific object that appears into its view. The UART hardware connections are then made between the Pixy Camera and the SJOne board. The SJOne board registers are initialized and the baud rate is set to match the baud rate of the Pixy Camera. The task then continuously runs the read function in order to continuously grab data from the Pixy Camera. The SJOne board then continuously checks to see if the received data from the Pixy Camera matches the sync byte 0xaa55. This tells the SJOne board that the camera has picked up on an object. Once this byte is received the following bytes will include information regarding the recognized object. Below is the flow of the UART communication, as well as the Pseudo Code behind the implementation.

[[File: Pixy to SJOne board UART flowchart.png| border|400px|center| <center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>]]

<center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>

''Implementation Pseudo Code''

init(Baud_Rate); // initialize Baud Rate and UART registers
while(1){
data_byte1 = read_byte();
data_byte2 = read_byte();
if (data_byte1 & data_byte2 == 0xaa55){ // 0xaa55 means Pixy has recognized the object and the following 12bytes sent will describe that object.
for (i < 11) // Allows the following 12 bytes of object information to be grabbed
grab_object_data = read_byte();
}
}

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion, we were able to complete necessary for our project. All though we done what was needed, we believe that it isn't enough and we could've have done more. When we were testing we ran into problems like every other group, but we able to navigate past them successfully. We are glad we took this Embedded Systems class, because we able to learn how FREE RTOS operating system works and how we can implement it for our project. We have also learned how to implement different types of communication protocols and how to program their drivers. The main part we learned from this project is to make sure to set a team leader and make sure people follow their lead to finish the tasks. Within a group project, you have to hold your teammates accountable, make sure you set deadlines, make sure people are getting their work done and this is pretty important in real work place situations.

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

https://youtu.be/OeIFKckJA0c

=== Project Source Code ===
https://github.com/nsarras/Object_Detector

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.
*http://cmucam.org/projects/cmucam5/wiki/Pixy_Serial_Protocol
*http://cmucam.org/projects/cmucam5/wiki/Porting_Guide
*https://learn.sparkfun.com/tutorials/pulse-width-modulation
*http://www.socialledge.com/sjsu/index.php?title=Realtime_OS_on_Embedded_Systems

File:CmpE146 F16 T5 BaudRate.png

2016-12-21T17:06:12Z

146 user5: Baud Rate Calculation

Baud Rate Calculation

F16: Object Detector

2016-12-21T16:39:29Z

146 user5: /* Design & Implementation */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: CmpE146 F16 T5 Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center>]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: CmpE146_F16_T5_Pixy_Camera.jpg| frame | center|<center>'''Figure 2. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File:CmpE146_F16_T5_Pixy_Camera_Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: CmpE146_F16_T5_Servo_Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receiving and sending bytes of information. Below is the Pseudocode an explanation for each function utilized within the UART driver.

<center><big> ''Initialization''</big> </center>

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below. From this equation we derive the DLL and DLM registers which are used as the divisors to establish the desired baud rate for the data transfer. The DLAB bit must also be initialized at is used as both the Receiver Buffer Register and the Transmit Holding Register. Both of which are set at the LCR register. The LCR register is also responsible for setting the start and stop bits of the UART communication.

[[File: Baud Rate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

''Initialization PseudoCode''

PCONP = Power on UART2
PCLKSEL1 = Set Peripheral Clock
PINSEL4 = Set RX and TX pin functionalities
// Acquire DLL_Value from desired Baud Rate
DLL_Value = clock/(16*Baud)
Set DLM and DLL registers according to DLL_Value
LCR = Set 8 bit data transfer Length
LCR = Set start and stop bit for data transfer

<center><big> ''Receiving Data''</big> </center>

Since the SJOne board is only receiving data from the Pixy Camera and not sending data in this situation only a receive byte function is required. The function waits to see if the data is transmitted from the Pixy Camera to the SJOne board by checking the RDR (Receiver Data Ready) register. Once the value of this register is set to 1, the receive function can then capture the transmitted data by looking at the RBR (Receiver Buffer Register).

''Receiving Function PseudoCode''

while (1){
if(RDR){
break;
}
}
new_data = RBR;

=== Pixy Camera Software Implementation ===

The Pixy Camera is able to externally communicate with micro-controllers by sending data regarding the object it is tracking, as well as receiving data for controlling the Pixy Camera itself. When the Pixy camera locates a recognized or preprogrammed object it begins to send information to a connected peripheral in bytes regarding that object. The first few bytes the Pixy camera sends are sync bytes, in other words bytes that let the external device know that the Pixy Camera has picked up on an object. The rest of the following bytes indicate the signature of the object, where it is on the screen with respect to the camera view (x/y center), as well as the width and the height of the object. Below are the specific bytes the Pixy Camera send to the SJOne board when it picks up on a device.

[[File: Pixy_ObjectBlock.png| frame | center| <center> '''Figure 7. Pixy Camera Object Block Data transfer''' </center>]]

The SJOne board is also able to control information to the Pixy Camera in order to control the pan and tilt of the servos, LED color, and camera brightness. However, the pan and tilt byte information is our area of concern with respect to control information being sent to the Pixy Camera. If the correct bytes are sent then the SJOne board acts as a master driving the pan and tilt servos of the Pixy Camera.

[[File: Pan & Tilt-Servo Control.png| frame | center| <center> '''Figure 8. Pan & Tilt Byte information''' </center>]]

=== Implementation ===

Initially, the Pixy Camera is programmed to recognize a specific object that appears into its view. The UART hardware connections are then made between the Pixy Camera and the SJOne board. The SJOne board registers are initialized and the baud rate is set to match the baud rate of the Pixy Camera. The task then continuously runs the read function in order to continuously grab data from the Pixy Camera. The SJOne board then continuously checks to see if the received data from the Pixy Camera matches the sync byte 0xaa55. This tells the SJOne board that the camera has picked up on an object. Once this byte is received the following bytes will include information regarding the recognized object. Below is the flow of the UART communication, as well as the Pseudo Code behind the implementation.

[[File: Pixy to SJOne board UART flowchart.png| border|400px|center| <center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>]]

<center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>

''Implementation Pseudo Code''

init(Baud_Rate); // initialize Baud Rate and UART registers
while(1){
data_byte1 = read_byte();
data_byte2 = read_byte();
if (data_byte1 & data_byte2 == 0xaa55){ // 0xaa55 means Pixy has recognized the object and the following 12bytes sent will describe that object.
for (i < 11) // Allows the following 12 bytes of object information to be grabbed
grab_object_data = read_byte();
}
}

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion, we were able to complete necessary for our project. All though we done what was needed, we believe that it isn't enough and we could've have done more. When we were testing we ran into problems like every other group, but we able to navigate past them successfully. We are glad we took this Embedded Systems class, because we able to learn how FREE RTOS operating system works and how we can implement it for our project. We have also learned how to implement different types of communication protocols and how to program their drivers. The main part we learned from this project is to make sure to set a team leader and make sure people follow their lead to finish the tasks. Within a group project, you have to hold your teammates accountable, make sure you set deadlines, make sure people are getting their work done and this is pretty important in real work place situations.

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

https://youtu.be/gaMJuxoMcQs

=== Project Source Code ===
https://github.com/nsarras/Object_Detector

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.
*http://cmucam.org/projects/cmucam5/wiki/Pixy_Serial_Protocol
*http://cmucam.org/projects/cmucam5/wiki/Porting_Guide
*https://learn.sparkfun.com/tutorials/pulse-width-modulation
*http://www.socialledge.com/sjsu/index.php?title=Realtime_OS_on_Embedded_Systems

File:CmpE146 F16 T5 Servo Motor.jpg

2016-12-21T16:37:38Z

146 user5: CmpE146_F16_T5_Servo_Motor.jpg

CmpE146_F16_T5_Servo_Motor.jpg

File:CmpE146 F16 T5 Pixy Camera Mount.jpg

2016-12-21T16:36:48Z

146 user5: CmpE146_F16_T5_Pixy_Camera_Mount.jpg

CmpE146_F16_T5_Pixy_Camera_Mount.jpg

File:CmpE146 F16 T5 Pixy Camera.jpg

2016-12-21T16:35:01Z

146 user5: CmpE146_F16_T5_Pixy_Camera.jpg

CmpE146_F16_T5_Pixy_Camera.jpg

File:CmpE146 F16 T5 Hardware Diagram.png

2016-12-21T16:31:13Z

146 user5: Hardware interface between SJOne Board, Pixy Camera, and the PWM servos.

Hardware interface between SJOne Board, Pixy Camera, and the PWM servos.

F16: Object Detector

2016-12-21T01:59:37Z

146 user5: /* Project Source Code */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center>]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: Pixy Camera.jpg| frame | center|<center>'''Figure 2. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File: Pixy Camera Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: Servo Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receiving and sending bytes of information. Below is the Pseudocode an explanation for each function utilized within the UART driver.

<center><big> ''Initialization''</big> </center>

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below. From this equation we derive the DLL and DLM registers which are used as the divisors to establish the desired baud rate for the data transfer. The DLAB bit must also be initialized at is used as both the Receiver Buffer Register and the Transmit Holding Register. Both of which are set at the LCR register. The LCR register is also responsible for setting the start and stop bits of the UART communication.

[[File: Baud Rate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

''Initialization PseudoCode''

PCONP = Power on UART2
PCLKSEL1 = Set Peripheral Clock
PINSEL4 = Set RX and TX pin functionalities
// Acquire DLL_Value from desired Baud Rate
DLL_Value = clock/(16*Baud)
Set DLM and DLL registers according to DLL_Value
LCR = Set 8 bit data transfer Length
LCR = Set start and stop bit for data transfer

<center><big> ''Receiving Data''</big> </center>

Since the SJOne board is only receiving data from the Pixy Camera and not sending data in this situation only a receive byte function is required. The function waits to see if the data is transmitted from the Pixy Camera to the SJOne board by checking the RDR (Receiver Data Ready) register. Once the value of this register is set to 1, the receive function can then capture the transmitted data by looking at the RBR (Receiver Buffer Register).

''Receiving Function PseudoCode''

while (1){
if(RDR){
break;
}
}
new_data = RBR;

=== Pixy Camera Software Implementation ===

The Pixy Camera is able to externally communicate with micro-controllers by sending data regarding the object it is tracking, as well as receiving data for controlling the Pixy Camera itself. When the Pixy camera locates a recognized or preprogrammed object it begins to send information to a connected peripheral in bytes regarding that object. The first few bytes the Pixy camera sends are sync bytes, in other words bytes that let the external device know that the Pixy Camera has picked up on an object. The rest of the following bytes indicate the signature of the object, where it is on the screen with respect to the camera view (x/y center), as well as the width and the height of the object. Below are the specific bytes the Pixy Camera send to the SJOne board when it picks up on a device.

[[File: Pixy_ObjectBlock.png| frame | center| <center> '''Figure 7. Pixy Camera Object Block Data transfer''' </center>]]

The SJOne board is also able to control information to the Pixy Camera in order to control the pan and tilt of the servos, LED color, and camera brightness. However, the pan and tilt byte information is our area of concern with respect to control information being sent to the Pixy Camera. If the correct bytes are sent then the SJOne board acts as a master driving the pan and tilt servos of the Pixy Camera.

[[File: Pan & Tilt-Servo Control.png| frame | center| <center> '''Figure 8. Pan & Tilt Byte information''' </center>]]

=== Implementation ===

Initially, the Pixy Camera is programmed to recognize a specific object that appears into its view. The UART hardware connections are then made between the Pixy Camera and the SJOne board. The SJOne board registers are initialized and the baud rate is set to match the baud rate of the Pixy Camera. The task then continuously runs the read function in order to continuously grab data from the Pixy Camera. The SJOne board then continuously checks to see if the received data from the Pixy Camera matches the sync byte 0xaa55. This tells the SJOne board that the camera has picked up on an object. Once this byte is received the following bytes will include information regarding the recognized object. Below is the flow of the UART communication, as well as the Pseudo Code behind the implementation.

[[File: Pixy to SJOne board UART flowchart.png| border|400px|center| <center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>]]

<center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>

''Implementation Pseudo Code''

init(Baud_Rate); // initialize Baud Rate and UART registers
while(1){
data_byte1 = read_byte();
data_byte2 = read_byte();
if (data_byte1 & data_byte2 == 0xaa55){ // 0xaa55 means Pixy has recognized the object and the following 12bytes sent will describe that object.
for (i < 11) // Allows the following 12 bytes of object information to be grabbed
grab_object_data = read_byte();
}
}

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion, we were able to complete necessary for our project. All though we done what was needed, we believe that it isn't enough and we could've have done more. When we were testing we ran into problems like every other group, but we able to navigate past them successfully. We are glad we took this Embedded Systems class, because we able to learn how FREE RTOS operating system works and how we can implement it for our project. We have also learned how to implement different types of communication protocols and how to program their drivers. The main part we learned from this project is to make sure to set a team leader and make sure people follow their lead to finish the tasks. Within a group project, you have to hold your teammates accountable, make sure you set deadlines, make sure people are getting their work done and this is pretty important in real work place situations.

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

https://youtu.be/gaMJuxoMcQs

=== Project Source Code ===
https://github.com/nsarras/Object_Detector

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.
*http://cmucam.org/projects/cmucam5/wiki/Pixy_Serial_Protocol
*http://cmucam.org/projects/cmucam5/wiki/Porting_Guide
*https://learn.sparkfun.com/tutorials/pulse-width-modulation
*http://www.socialledge.com/sjsu/index.php?title=Realtime_OS_on_Embedded_Systems

F16: Object Detector

2016-12-21T01:58:32Z

146 user5: /* Project Video */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center>]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: Pixy Camera.jpg| frame | center|<center>'''Figure 2. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File: Pixy Camera Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: Servo Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receiving and sending bytes of information. Below is the Pseudocode an explanation for each function utilized within the UART driver.

<center><big> ''Initialization''</big> </center>

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below. From this equation we derive the DLL and DLM registers which are used as the divisors to establish the desired baud rate for the data transfer. The DLAB bit must also be initialized at is used as both the Receiver Buffer Register and the Transmit Holding Register. Both of which are set at the LCR register. The LCR register is also responsible for setting the start and stop bits of the UART communication.

[[File: Baud Rate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

''Initialization PseudoCode''

PCONP = Power on UART2
PCLKSEL1 = Set Peripheral Clock
PINSEL4 = Set RX and TX pin functionalities
// Acquire DLL_Value from desired Baud Rate
DLL_Value = clock/(16*Baud)
Set DLM and DLL registers according to DLL_Value
LCR = Set 8 bit data transfer Length
LCR = Set start and stop bit for data transfer

<center><big> ''Receiving Data''</big> </center>

Since the SJOne board is only receiving data from the Pixy Camera and not sending data in this situation only a receive byte function is required. The function waits to see if the data is transmitted from the Pixy Camera to the SJOne board by checking the RDR (Receiver Data Ready) register. Once the value of this register is set to 1, the receive function can then capture the transmitted data by looking at the RBR (Receiver Buffer Register).

''Receiving Function PseudoCode''

while (1){
if(RDR){
break;
}
}
new_data = RBR;

=== Pixy Camera Software Implementation ===

The Pixy Camera is able to externally communicate with micro-controllers by sending data regarding the object it is tracking, as well as receiving data for controlling the Pixy Camera itself. When the Pixy camera locates a recognized or preprogrammed object it begins to send information to a connected peripheral in bytes regarding that object. The first few bytes the Pixy camera sends are sync bytes, in other words bytes that let the external device know that the Pixy Camera has picked up on an object. The rest of the following bytes indicate the signature of the object, where it is on the screen with respect to the camera view (x/y center), as well as the width and the height of the object. Below are the specific bytes the Pixy Camera send to the SJOne board when it picks up on a device.

[[File: Pixy_ObjectBlock.png| frame | center| <center> '''Figure 7. Pixy Camera Object Block Data transfer''' </center>]]

The SJOne board is also able to control information to the Pixy Camera in order to control the pan and tilt of the servos, LED color, and camera brightness. However, the pan and tilt byte information is our area of concern with respect to control information being sent to the Pixy Camera. If the correct bytes are sent then the SJOne board acts as a master driving the pan and tilt servos of the Pixy Camera.

[[File: Pan & Tilt-Servo Control.png| frame | center| <center> '''Figure 8. Pan & Tilt Byte information''' </center>]]

=== Implementation ===

Initially, the Pixy Camera is programmed to recognize a specific object that appears into its view. The UART hardware connections are then made between the Pixy Camera and the SJOne board. The SJOne board registers are initialized and the baud rate is set to match the baud rate of the Pixy Camera. The task then continuously runs the read function in order to continuously grab data from the Pixy Camera. The SJOne board then continuously checks to see if the received data from the Pixy Camera matches the sync byte 0xaa55. This tells the SJOne board that the camera has picked up on an object. Once this byte is received the following bytes will include information regarding the recognized object. Below is the flow of the UART communication, as well as the Pseudo Code behind the implementation.

[[File: Pixy to SJOne board UART flowchart.png| border|400px|center| <center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>]]

<center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>

''Implementation Pseudo Code''

init(Baud_Rate); // initialize Baud Rate and UART registers
while(1){
data_byte1 = read_byte();
data_byte2 = read_byte();
if (data_byte1 & data_byte2 == 0xaa55){ // 0xaa55 means Pixy has recognized the object and the following 12bytes sent will describe that object.
for (i < 11) // Allows the following 12 bytes of object information to be grabbed
grab_object_data = read_byte();
}
}

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion, we were able to complete necessary for our project. All though we done what was needed, we believe that it isn't enough and we could've have done more. When we were testing we ran into problems like every other group, but we able to navigate past them successfully. We are glad we took this Embedded Systems class, because we able to learn how FREE RTOS operating system works and how we can implement it for our project. We have also learned how to implement different types of communication protocols and how to program their drivers. The main part we learned from this project is to make sure to set a team leader and make sure people follow their lead to finish the tasks. Within a group project, you have to hold your teammates accountable, make sure you set deadlines, make sure people are getting their work done and this is pretty important in real work place situations.

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

https://youtu.be/gaMJuxoMcQs

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.
*http://cmucam.org/projects/cmucam5/wiki/Pixy_Serial_Protocol
*http://cmucam.org/projects/cmucam5/wiki/Porting_Guide
*https://learn.sparkfun.com/tutorials/pulse-width-modulation
*http://www.socialledge.com/sjsu/index.php?title=Realtime_OS_on_Embedded_Systems

F16: Object Detector

2016-12-21T01:58:11Z

146 user5: /* Project Video */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center>]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: Pixy Camera.jpg| frame | center|<center>'''Figure 2. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File: Pixy Camera Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: Servo Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receiving and sending bytes of information. Below is the Pseudocode an explanation for each function utilized within the UART driver.

<center><big> ''Initialization''</big> </center>

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below. From this equation we derive the DLL and DLM registers which are used as the divisors to establish the desired baud rate for the data transfer. The DLAB bit must also be initialized at is used as both the Receiver Buffer Register and the Transmit Holding Register. Both of which are set at the LCR register. The LCR register is also responsible for setting the start and stop bits of the UART communication.

[[File: Baud Rate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

''Initialization PseudoCode''

PCONP = Power on UART2
PCLKSEL1 = Set Peripheral Clock
PINSEL4 = Set RX and TX pin functionalities
// Acquire DLL_Value from desired Baud Rate
DLL_Value = clock/(16*Baud)
Set DLM and DLL registers according to DLL_Value
LCR = Set 8 bit data transfer Length
LCR = Set start and stop bit for data transfer

<center><big> ''Receiving Data''</big> </center>

Since the SJOne board is only receiving data from the Pixy Camera and not sending data in this situation only a receive byte function is required. The function waits to see if the data is transmitted from the Pixy Camera to the SJOne board by checking the RDR (Receiver Data Ready) register. Once the value of this register is set to 1, the receive function can then capture the transmitted data by looking at the RBR (Receiver Buffer Register).

''Receiving Function PseudoCode''

while (1){
if(RDR){
break;
}
}
new_data = RBR;

=== Pixy Camera Software Implementation ===

The Pixy Camera is able to externally communicate with micro-controllers by sending data regarding the object it is tracking, as well as receiving data for controlling the Pixy Camera itself. When the Pixy camera locates a recognized or preprogrammed object it begins to send information to a connected peripheral in bytes regarding that object. The first few bytes the Pixy camera sends are sync bytes, in other words bytes that let the external device know that the Pixy Camera has picked up on an object. The rest of the following bytes indicate the signature of the object, where it is on the screen with respect to the camera view (x/y center), as well as the width and the height of the object. Below are the specific bytes the Pixy Camera send to the SJOne board when it picks up on a device.

[[File: Pixy_ObjectBlock.png| frame | center| <center> '''Figure 7. Pixy Camera Object Block Data transfer''' </center>]]

The SJOne board is also able to control information to the Pixy Camera in order to control the pan and tilt of the servos, LED color, and camera brightness. However, the pan and tilt byte information is our area of concern with respect to control information being sent to the Pixy Camera. If the correct bytes are sent then the SJOne board acts as a master driving the pan and tilt servos of the Pixy Camera.

[[File: Pan & Tilt-Servo Control.png| frame | center| <center> '''Figure 8. Pan & Tilt Byte information''' </center>]]

=== Implementation ===

Initially, the Pixy Camera is programmed to recognize a specific object that appears into its view. The UART hardware connections are then made between the Pixy Camera and the SJOne board. The SJOne board registers are initialized and the baud rate is set to match the baud rate of the Pixy Camera. The task then continuously runs the read function in order to continuously grab data from the Pixy Camera. The SJOne board then continuously checks to see if the received data from the Pixy Camera matches the sync byte 0xaa55. This tells the SJOne board that the camera has picked up on an object. Once this byte is received the following bytes will include information regarding the recognized object. Below is the flow of the UART communication, as well as the Pseudo Code behind the implementation.

[[File: Pixy to SJOne board UART flowchart.png| border|400px|center| <center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>]]

<center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>

''Implementation Pseudo Code''

init(Baud_Rate); // initialize Baud Rate and UART registers
while(1){
data_byte1 = read_byte();
data_byte2 = read_byte();
if (data_byte1 & data_byte2 == 0xaa55){ // 0xaa55 means Pixy has recognized the object and the following 12bytes sent will describe that object.
for (i < 11) // Allows the following 12 bytes of object information to be grabbed
grab_object_data = read_byte();
}
}

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion, we were able to complete necessary for our project. All though we done what was needed, we believe that it isn't enough and we could've have done more. When we were testing we ran into problems like every other group, but we able to navigate past them successfully. We are glad we took this Embedded Systems class, because we able to learn how FREE RTOS operating system works and how we can implement it for our project. We have also learned how to implement different types of communication protocols and how to program their drivers. The main part we learned from this project is to make sure to set a team leader and make sure people follow their lead to finish the tasks. Within a group project, you have to hold your teammates accountable, make sure you set deadlines, make sure people are getting their work done and this is pretty important in real work place situations.

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.
|https://youtu.be/gaMJuxoMcQs

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.
*http://cmucam.org/projects/cmucam5/wiki/Pixy_Serial_Protocol
*http://cmucam.org/projects/cmucam5/wiki/Porting_Guide
*https://learn.sparkfun.com/tutorials/pulse-width-modulation
*http://www.socialledge.com/sjsu/index.php?title=Realtime_OS_on_Embedded_Systems

F16: Object Detector

2016-12-21T01:57:50Z

146 user5: /* Project Video */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center>]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: Pixy Camera.jpg| frame | center|<center>'''Figure 2. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File: Pixy Camera Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: Servo Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receiving and sending bytes of information. Below is the Pseudocode an explanation for each function utilized within the UART driver.

<center><big> ''Initialization''</big> </center>

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below. From this equation we derive the DLL and DLM registers which are used as the divisors to establish the desired baud rate for the data transfer. The DLAB bit must also be initialized at is used as both the Receiver Buffer Register and the Transmit Holding Register. Both of which are set at the LCR register. The LCR register is also responsible for setting the start and stop bits of the UART communication.

[[File: Baud Rate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

''Initialization PseudoCode''

PCONP = Power on UART2
PCLKSEL1 = Set Peripheral Clock
PINSEL4 = Set RX and TX pin functionalities
// Acquire DLL_Value from desired Baud Rate
DLL_Value = clock/(16*Baud)
Set DLM and DLL registers according to DLL_Value
LCR = Set 8 bit data transfer Length
LCR = Set start and stop bit for data transfer

<center><big> ''Receiving Data''</big> </center>

Since the SJOne board is only receiving data from the Pixy Camera and not sending data in this situation only a receive byte function is required. The function waits to see if the data is transmitted from the Pixy Camera to the SJOne board by checking the RDR (Receiver Data Ready) register. Once the value of this register is set to 1, the receive function can then capture the transmitted data by looking at the RBR (Receiver Buffer Register).

''Receiving Function PseudoCode''

while (1){
if(RDR){
break;
}
}
new_data = RBR;

=== Pixy Camera Software Implementation ===

The Pixy Camera is able to externally communicate with micro-controllers by sending data regarding the object it is tracking, as well as receiving data for controlling the Pixy Camera itself. When the Pixy camera locates a recognized or preprogrammed object it begins to send information to a connected peripheral in bytes regarding that object. The first few bytes the Pixy camera sends are sync bytes, in other words bytes that let the external device know that the Pixy Camera has picked up on an object. The rest of the following bytes indicate the signature of the object, where it is on the screen with respect to the camera view (x/y center), as well as the width and the height of the object. Below are the specific bytes the Pixy Camera send to the SJOne board when it picks up on a device.

[[File: Pixy_ObjectBlock.png| frame | center| <center> '''Figure 7. Pixy Camera Object Block Data transfer''' </center>]]

The SJOne board is also able to control information to the Pixy Camera in order to control the pan and tilt of the servos, LED color, and camera brightness. However, the pan and tilt byte information is our area of concern with respect to control information being sent to the Pixy Camera. If the correct bytes are sent then the SJOne board acts as a master driving the pan and tilt servos of the Pixy Camera.

[[File: Pan & Tilt-Servo Control.png| frame | center| <center> '''Figure 8. Pan & Tilt Byte information''' </center>]]

=== Implementation ===

Initially, the Pixy Camera is programmed to recognize a specific object that appears into its view. The UART hardware connections are then made between the Pixy Camera and the SJOne board. The SJOne board registers are initialized and the baud rate is set to match the baud rate of the Pixy Camera. The task then continuously runs the read function in order to continuously grab data from the Pixy Camera. The SJOne board then continuously checks to see if the received data from the Pixy Camera matches the sync byte 0xaa55. This tells the SJOne board that the camera has picked up on an object. Once this byte is received the following bytes will include information regarding the recognized object. Below is the flow of the UART communication, as well as the Pseudo Code behind the implementation.

[[File: Pixy to SJOne board UART flowchart.png| border|400px|center| <center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>]]

<center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>

''Implementation Pseudo Code''

init(Baud_Rate); // initialize Baud Rate and UART registers
while(1){
data_byte1 = read_byte();
data_byte2 = read_byte();
if (data_byte1 & data_byte2 == 0xaa55){ // 0xaa55 means Pixy has recognized the object and the following 12bytes sent will describe that object.
for (i < 11) // Allows the following 12 bytes of object information to be grabbed
grab_object_data = read_byte();
}
}

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion, we were able to complete necessary for our project. All though we done what was needed, we believe that it isn't enough and we could've have done more. When we were testing we ran into problems like every other group, but we able to navigate past them successfully. We are glad we took this Embedded Systems class, because we able to learn how FREE RTOS operating system works and how we can implement it for our project. We have also learned how to implement different types of communication protocols and how to program their drivers. The main part we learned from this project is to make sure to set a team leader and make sure people follow their lead to finish the tasks. Within a group project, you have to hold your teammates accountable, make sure you set deadlines, make sure people are getting their work done and this is pretty important in real work place situations.

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.
https://youtu.be/gaMJuxoMcQs

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.
*http://cmucam.org/projects/cmucam5/wiki/Pixy_Serial_Protocol
*http://cmucam.org/projects/cmucam5/wiki/Porting_Guide
*https://learn.sparkfun.com/tutorials/pulse-width-modulation
*http://www.socialledge.com/sjsu/index.php?title=Realtime_OS_on_Embedded_Systems

F16: Object Detector

2016-12-20T23:33:38Z

146 user5: /* Implementation */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center>]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: Pixy Camera.jpg| frame | center|<center>'''Figure 2. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File: Pixy Camera Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: Servo Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receiving and sending bytes of information. Below is the Pseudocode an explanation for each function utilized within the UART driver.

<center><big> ''Initialization''</big> </center>

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below. From this equation we derive the DLL and DLM registers which are used as the divisors to establish the desired baud rate for the data transfer. The DLAB bit must also be initialized at is used as both the Receiver Buffer Register and the Transmit Holding Register. Both of which are set at the LCR register. The LCR register is also responsible for setting the start and stop bits of the UART communication.

[[File: Baud Rate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

''Initialization PseudoCode''

PCONP = Power on UART2
PCLKSEL1 = Set Peripheral Clock
PINSEL4 = Set RX and TX pin functionalities
// Acquire DLL_Value from desired Baud Rate
DLL_Value = clock/(16*Baud)
Set DLM and DLL registers according to DLL_Value
LCR = Set 8 bit data transfer Length
LCR = Set start and stop bit for data transfer

<center><big> ''Receiving Data''</big> </center>

Since the SJOne board is only receiving data from the Pixy Camera and not sending data in this situation only a receive byte function is required. The function waits to see if the data is transmitted from the Pixy Camera to the SJOne board by checking the RDR (Receiver Data Ready) register. Once the value of this register is set to 1, the receive function can then capture the transmitted data by looking at the RBR (Receiver Buffer Register).

''Receiving Function PseudoCode''

while (1){
if(RDR){
break;
}
}
new_data = RBR;

=== Pixy Camera Software Implementation ===

The Pixy Camera is able to externally communicate with micro-controllers by sending data regarding the object it is tracking, as well as receiving data for controlling the Pixy Camera itself. When the Pixy camera locates a recognized or preprogrammed object it begins to send information to a connected peripheral in bytes regarding that object. The first few bytes the Pixy camera sends are sync bytes, in other words bytes that let the external device know that the Pixy Camera has picked up on an object. The rest of the following bytes indicate the signature of the object, where it is on the screen with respect to the camera view (x/y center), as well as the width and the height of the object. Below are the specific bytes the Pixy Camera send to the SJOne board when it picks up on a device.

[[File: Pixy_ObjectBlock.png| frame | center| <center> '''Figure 7. Pixy Camera Object Block Data transfer''' </center>]]

The SJOne board is also able to control information to the Pixy Camera in order to control the pan and tilt of the servos, LED color, and camera brightness. However, the pan and tilt byte information is our area of concern with respect to control information being sent to the Pixy Camera. If the correct bytes are sent then the SJOne board acts as a master driving the pan and tilt servos of the Pixy Camera.

[[File: Pan & Tilt-Servo Control.png| frame | center| <center> '''Figure 8. Pan & Tilt Byte information''' </center>]]

=== Implementation ===

Initially, the Pixy Camera is programmed to recognize a specific object that appears into its view. The UART hardware connections are then made between the Pixy Camera and the SJOne board. The SJOne board registers are initialized and the baud rate is set to match the baud rate of the Pixy Camera. The task then continuously runs the read function in order to continuously grab data from the Pixy Camera. The SJOne board then continuously checks to see if the received data from the Pixy Camera matches the sync byte 0xaa55. This tells the SJOne board that the camera has picked up on an object. Once this byte is received the following bytes will include information regarding the recognized object. Below is the flow of the UART communication, as well as the Pseudo Code behind the implementation.

[[File: Pixy to SJOne board UART flowchart.png| border|400px|center| <center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>]]

<center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>

''Implementation Pseudo Code''

init(Baud_Rate); // initialize Baud Rate and UART registers
while(1){
data_byte1 = read_byte();
data_byte2 = read_byte();
if (data_byte1 & data_byte2 == 0xaa55){ // 0xaa55 means Pixy has recognized the object and the following 12bytes sent will describe that object.
for (i < 11) // Allows the following 12 bytes of object information to be grabbed
grab_object_data = read_byte();
}
}

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion, we were able to complete necessary for our project. All though we done what was needed, we believe that it isn't enough and we could've have done more. When we were testing we ran into problems like every other group, but we able to navigate past them successfully. We are glad we took this Embedded Systems class, because we able to learn how FREE RTOS operating system works and how we can implement it for our project. We have also learned how to implement different types of communication protocols and how to program their drivers. The main part we learned from this project is to make sure to set a team leader and make sure people follow their lead to finish the tasks. Within a group project, you have to hold your teammates accountable, make sure you set deadlines, make sure people are getting their work done and this is pretty important in real work place situations.

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.
*http://cmucam.org/projects/cmucam5/wiki/Pixy_Serial_Protocol
*http://cmucam.org/projects/cmucam5/wiki/Porting_Guide
*https://learn.sparkfun.com/tutorials/pulse-width-modulation
*http://www.socialledge.com/sjsu/index.php?title=Realtime_OS_on_Embedded_Systems

F16: Object Detector

2016-12-20T23:32:21Z

146 user5: /* Implementation */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center>]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: Pixy Camera.jpg| frame | center|<center>'''Figure 2. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File: Pixy Camera Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: Servo Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receiving and sending bytes of information. Below is the Pseudocode an explanation for each function utilized within the UART driver.

<center><big> ''Initialization''</big> </center>

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below. From this equation we derive the DLL and DLM registers which are used as the divisors to establish the desired baud rate for the data transfer. The DLAB bit must also be initialized at is used as both the Receiver Buffer Register and the Transmit Holding Register. Both of which are set at the LCR register. The LCR register is also responsible for setting the start and stop bits of the UART communication.

[[File: Baud Rate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

''Initialization PseudoCode''

PCONP = Power on UART2
PCLKSEL1 = Set Peripheral Clock
PINSEL4 = Set RX and TX pin functionalities
// Acquire DLL_Value from desired Baud Rate
DLL_Value = clock/(16*Baud)
Set DLM and DLL registers according to DLL_Value
LCR = Set 8 bit data transfer Length
LCR = Set start and stop bit for data transfer

<center><big> ''Receiving Data''</big> </center>

Since the SJOne board is only receiving data from the Pixy Camera and not sending data in this situation only a receive byte function is required. The function waits to see if the data is transmitted from the Pixy Camera to the SJOne board by checking the RDR (Receiver Data Ready) register. Once the value of this register is set to 1, the receive function can then capture the transmitted data by looking at the RBR (Receiver Buffer Register).

''Receiving Function PseudoCode''

while (1){
if(RDR){
break;
}
}
new_data = RBR;

=== Pixy Camera Software Implementation ===

The Pixy Camera is able to externally communicate with micro-controllers by sending data regarding the object it is tracking, as well as receiving data for controlling the Pixy Camera itself. When the Pixy camera locates a recognized or preprogrammed object it begins to send information to a connected peripheral in bytes regarding that object. The first few bytes the Pixy camera sends are sync bytes, in other words bytes that let the external device know that the Pixy Camera has picked up on an object. The rest of the following bytes indicate the signature of the object, where it is on the screen with respect to the camera view (x/y center), as well as the width and the height of the object. Below are the specific bytes the Pixy Camera send to the SJOne board when it picks up on a device.

[[File: Pixy_ObjectBlock.png| frame | center| <center> '''Figure 7. Pixy Camera Object Block Data transfer''' </center>]]

The SJOne board is also able to control information to the Pixy Camera in order to control the pan and tilt of the servos, LED color, and camera brightness. However, the pan and tilt byte information is our area of concern with respect to control information being sent to the Pixy Camera. If the correct bytes are sent then the SJOne board acts as a master driving the pan and tilt servos of the Pixy Camera.

[[File: Pan & Tilt-Servo Control.png| frame | center| <center> '''Figure 8. Pan & Tilt Byte information''' </center>]]

=== Implementation ===

Initially, the Pixy Camera is programmed to recognize a specific object that appears into its view. The UART hardware connections are then made between the Pixy Camera and the SJOne board. The SJOne board registers are initialized and the baud rate is set to match the baud rate of the Pixy Camera. The task then continuously runs the read function in order to continuously grab data from the Pixy Camera. The SJOne board then continuously checks to see if the received data from the Pixy Camera matches the sync byte 0xaa55. This tells the SJOne board that the camera has picked up on an object. Once this byte is received the following bytes will include information regarding the recognized object. Below is the flow of the UART communication, as well as the Pseudo Code behind the implementation.

[[File: Pixy to SJOne board UART flowchart.png| border|400px|center| <center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>]]

<center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>

''Implementation Pseudo Code''

init(Baud_Rate); // initialize Baud Rate and UART registers
while(1){
data_byte1 = read_byte();
data_byte2 = read_byte();

if (data_byte1 & data_byte2 == 0xaa55){ //When 0xaa55 is grabbed the Pixy has recognized the object and the following 12bytes of information sent will describe that object.
for (i < 11) // Allows the following 12 bytes of object information to be grabbed
grab_object_data = read_byte();
}

}

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion, we were able to complete necessary for our project. All though we done what was needed, we believe that it isn't enough and we could've have done more. When we were testing we ran into problems like every other group, but we able to navigate past them successfully. We are glad we took this Embedded Systems class, because we able to learn how FREE RTOS operating system works and how we can implement it for our project. We have also learned how to implement different types of communication protocols and how to program their drivers. The main part we learned from this project is to make sure to set a team leader and make sure people follow their lead to finish the tasks. Within a group project, you have to hold your teammates accountable, make sure you set deadlines, make sure people are getting their work done and this is pretty important in real work place situations.

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.
*http://cmucam.org/projects/cmucam5/wiki/Pixy_Serial_Protocol
*http://cmucam.org/projects/cmucam5/wiki/Porting_Guide
*https://learn.sparkfun.com/tutorials/pulse-width-modulation
*http://www.socialledge.com/sjsu/index.php?title=Realtime_OS_on_Embedded_Systems

F16: Object Detector

2016-12-20T23:21:24Z

146 user5: /* Implementation */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center>]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: Pixy Camera.jpg| frame | center|<center>'''Figure 2. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File: Pixy Camera Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: Servo Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receiving and sending bytes of information. Below is the Pseudocode an explanation for each function utilized within the UART driver.

<center><big> ''Initialization''</big> </center>

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below. From this equation we derive the DLL and DLM registers which are used as the divisors to establish the desired baud rate for the data transfer. The DLAB bit must also be initialized at is used as both the Receiver Buffer Register and the Transmit Holding Register. Both of which are set at the LCR register. The LCR register is also responsible for setting the start and stop bits of the UART communication.

[[File: Baud Rate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

''Initialization PseudoCode''

PCONP = Power on UART2
PCLKSEL1 = Set Peripheral Clock
PINSEL4 = Set RX and TX pin functionalities
// Acquire DLL_Value from desired Baud Rate
DLL_Value = clock/(16*Baud)
Set DLM and DLL registers according to DLL_Value
LCR = Set 8 bit data transfer Length
LCR = Set start and stop bit for data transfer

<center><big> ''Receiving Data''</big> </center>

Since the SJOne board is only receiving data from the Pixy Camera and not sending data in this situation only a receive byte function is required. The function waits to see if the data is transmitted from the Pixy Camera to the SJOne board by checking the RDR (Receiver Data Ready) register. Once the value of this register is set to 1, the receive function can then capture the transmitted data by looking at the RBR (Receiver Buffer Register).

''Receiving Function PseudoCode''

while (1){
if(RDR){
break;
}
}
new_data = RBR;

=== Pixy Camera Software Implementation ===

The Pixy Camera is able to externally communicate with micro-controllers by sending data regarding the object it is tracking, as well as receiving data for controlling the Pixy Camera itself. When the Pixy camera locates a recognized or preprogrammed object it begins to send information to a connected peripheral in bytes regarding that object. The first few bytes the Pixy camera sends are sync bytes, in other words bytes that let the external device know that the Pixy Camera has picked up on an object. The rest of the following bytes indicate the signature of the object, where it is on the screen with respect to the camera view (x/y center), as well as the width and the height of the object. Below are the specific bytes the Pixy Camera send to the SJOne board when it picks up on a device.

[[File: Pixy_ObjectBlock.png| frame | center| <center> '''Figure 7. Pixy Camera Object Block Data transfer''' </center>]]

The SJOne board is also able to control information to the Pixy Camera in order to control the pan and tilt of the servos, LED color, and camera brightness. However, the pan and tilt byte information is our area of concern with respect to control information being sent to the Pixy Camera. If the correct bytes are sent then the SJOne board acts as a master driving the pan and tilt servos of the Pixy Camera.

[[File: Pan & Tilt-Servo Control.png| frame | center| <center> '''Figure 8. Pan & Tilt Byte information''' </center>]]

=== Implementation ===

Initially, the Pixy Camera is programmed to recognize a specific object that appears into its view. The UART hardware connections are then made between the Pixy Camera and the SJOne board. The SJOne board registers are initialized and the baud rate is set to match the baud rate of the Pixy Camera. The task then continuously runs the read function in order to continuously grab data from the Pixy Camera. The SJOne board then continuously checks to see if the received data from the Pixy Camera matches the sync byte 0xaa55. This tells the SJOne board that the camera has picked up on an object. Once this byte is received the following bytes will include information regarding the recognized object. Below is the flow of the UART communication, as well as the Pseudo Code behind the implementation.

[[File: Pixy to SJOne board UART flowchart.png| border|400px|center| <center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>]]

<center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion, we were able to complete necessary for our project. All though we done what was needed, we believe that it isn't enough and we could've have done more. When we were testing we ran into problems like every other group, but we able to navigate past them successfully. We are glad we took this Embedded Systems class, because we able to learn how FREE RTOS operating system works and how we can implement it for our project. We have also learned how to implement different types of communication protocols and how to program their drivers. The main part we learned from this project is to make sure to set a team leader and make sure people follow their lead to finish the tasks. Within a group project, you have to hold your teammates accountable, make sure you set deadlines, make sure people are getting their work done and this is pretty important in real work place situations.

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.
*http://cmucam.org/projects/cmucam5/wiki/Pixy_Serial_Protocol
*http://cmucam.org/projects/cmucam5/wiki/Porting_Guide
*https://learn.sparkfun.com/tutorials/pulse-width-modulation
*http://www.socialledge.com/sjsu/index.php?title=Realtime_OS_on_Embedded_Systems

F16: Object Detector

2016-12-20T23:20:44Z

146 user5: /* Implementation */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center>]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: Pixy Camera.jpg| frame | center|<center>'''Figure 2. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File: Pixy Camera Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: Servo Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receiving and sending bytes of information. Below is the Pseudocode an explanation for each function utilized within the UART driver.

<center><big> ''Initialization''</big> </center>

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below. From this equation we derive the DLL and DLM registers which are used as the divisors to establish the desired baud rate for the data transfer. The DLAB bit must also be initialized at is used as both the Receiver Buffer Register and the Transmit Holding Register. Both of which are set at the LCR register. The LCR register is also responsible for setting the start and stop bits of the UART communication.

[[File: Baud Rate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

''Initialization PseudoCode''

PCONP = Power on UART2
PCLKSEL1 = Set Peripheral Clock
PINSEL4 = Set RX and TX pin functionalities
// Acquire DLL_Value from desired Baud Rate
DLL_Value = clock/(16*Baud)
Set DLM and DLL registers according to DLL_Value
LCR = Set 8 bit data transfer Length
LCR = Set start and stop bit for data transfer

<center><big> ''Receiving Data''</big> </center>

Since the SJOne board is only receiving data from the Pixy Camera and not sending data in this situation only a receive byte function is required. The function waits to see if the data is transmitted from the Pixy Camera to the SJOne board by checking the RDR (Receiver Data Ready) register. Once the value of this register is set to 1, the receive function can then capture the transmitted data by looking at the RBR (Receiver Buffer Register).

''Receiving Function PseudoCode''

while (1){
if(RDR){
break;
}
}
new_data = RBR;

=== Pixy Camera Software Implementation ===

The Pixy Camera is able to externally communicate with micro-controllers by sending data regarding the object it is tracking, as well as receiving data for controlling the Pixy Camera itself. When the Pixy camera locates a recognized or preprogrammed object it begins to send information to a connected peripheral in bytes regarding that object. The first few bytes the Pixy camera sends are sync bytes, in other words bytes that let the external device know that the Pixy Camera has picked up on an object. The rest of the following bytes indicate the signature of the object, where it is on the screen with respect to the camera view (x/y center), as well as the width and the height of the object. Below are the specific bytes the Pixy Camera send to the SJOne board when it picks up on a device.

[[File: Pixy_ObjectBlock.png| frame | center| <center> '''Figure 7. Pixy Camera Object Block Data transfer''' </center>]]

The SJOne board is also able to control information to the Pixy Camera in order to control the pan and tilt of the servos, LED color, and camera brightness. However, the pan and tilt byte information is our area of concern with respect to control information being sent to the Pixy Camera. If the correct bytes are sent then the SJOne board acts as a master driving the pan and tilt servos of the Pixy Camera.

[[File: Pan & Tilt-Servo Control.png| frame | center| <center> '''Figure 8. Pan & Tilt Byte information''' </center>]]

=== Implementation ===

Initially, the Pixy Camera is programmed to recognize a specific object that appears into its view. The UART hardware connections are then made between the Pixy Camera and the SJOne board. The SJOne board registers are initialized and the baud rate is set to match the baud rate of the Pixy Camera. The task then continuously runs the read function in order to continuously grab data from the Pixy Camera. The SJOne board then continuously checks to see if the received data from the Pixy Camera matches the sync byte 0xaa55. This tells the SJOne board that the camera has picked up on an object. Once this byte is received the following bytes will include information regarding the recognized object. Below is the flow of the UART communication, as well as the Pseudo Code behind the implementation.

[[File: Pixy to SJOne board UART flowchart.png| border|400px|center| <center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>]]

[[<center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>]]

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion, we were able to complete necessary for our project. All though we done what was needed, we believe that it isn't enough and we could've have done more. When we were testing we ran into problems like every other group, but we able to navigate past them successfully. We are glad we took this Embedded Systems class, because we able to learn how FREE RTOS operating system works and how we can implement it for our project. We have also learned how to implement different types of communication protocols and how to program their drivers. The main part we learned from this project is to make sure to set a team leader and make sure people follow their lead to finish the tasks. Within a group project, you have to hold your teammates accountable, make sure you set deadlines, make sure people are getting their work done and this is pretty important in real work place situations.

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.
*http://cmucam.org/projects/cmucam5/wiki/Pixy_Serial_Protocol
*http://cmucam.org/projects/cmucam5/wiki/Porting_Guide
*https://learn.sparkfun.com/tutorials/pulse-width-modulation
*http://www.socialledge.com/sjsu/index.php?title=Realtime_OS_on_Embedded_Systems

F16: Object Detector

2016-12-20T23:14:03Z

146 user5: /* Software Design SJOne Board */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center>]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: Pixy Camera.jpg| frame | center|<center>'''Figure 2. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File: Pixy Camera Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: Servo Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receiving and sending bytes of information. Below is the Pseudocode an explanation for each function utilized within the UART driver.

<center><big> ''Initialization''</big> </center>

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below. From this equation we derive the DLL and DLM registers which are used as the divisors to establish the desired baud rate for the data transfer. The DLAB bit must also be initialized at is used as both the Receiver Buffer Register and the Transmit Holding Register. Both of which are set at the LCR register. The LCR register is also responsible for setting the start and stop bits of the UART communication.

[[File: Baud Rate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

''Initialization PseudoCode''

PCONP = Power on UART2
PCLKSEL1 = Set Peripheral Clock
PINSEL4 = Set RX and TX pin functionalities
// Acquire DLL_Value from desired Baud Rate
DLL_Value = clock/(16*Baud)
Set DLM and DLL registers according to DLL_Value
LCR = Set 8 bit data transfer Length
LCR = Set start and stop bit for data transfer

<center><big> ''Receiving Data''</big> </center>

Since the SJOne board is only receiving data from the Pixy Camera and not sending data in this situation only a receive byte function is required. The function waits to see if the data is transmitted from the Pixy Camera to the SJOne board by checking the RDR (Receiver Data Ready) register. Once the value of this register is set to 1, the receive function can then capture the transmitted data by looking at the RBR (Receiver Buffer Register).

''Receiving Function PseudoCode''

while (1){
if(RDR){
break;
}
}
new_data = RBR;

=== Pixy Camera Software Implementation ===

The Pixy Camera is able to externally communicate with micro-controllers by sending data regarding the object it is tracking, as well as receiving data for controlling the Pixy Camera itself. When the Pixy camera locates a recognized or preprogrammed object it begins to send information to a connected peripheral in bytes regarding that object. The first few bytes the Pixy camera sends are sync bytes, in other words bytes that let the external device know that the Pixy Camera has picked up on an object. The rest of the following bytes indicate the signature of the object, where it is on the screen with respect to the camera view (x/y center), as well as the width and the height of the object. Below are the specific bytes the Pixy Camera send to the SJOne board when it picks up on a device.

[[File: Pixy_ObjectBlock.png| frame | center| <center> '''Figure 7. Pixy Camera Object Block Data transfer''' </center>]]

The SJOne board is also able to control information to the Pixy Camera in order to control the pan and tilt of the servos, LED color, and camera brightness. However, the pan and tilt byte information is our area of concern with respect to control information being sent to the Pixy Camera. If the correct bytes are sent then the SJOne board acts as a master driving the pan and tilt servos of the Pixy Camera.

[[File: Pan & Tilt-Servo Control.png| frame | center| <center> '''Figure 8. Pan & Tilt Byte information''' </center>]]

=== Implementation ===

Initially, the Pixy Camera is programmed to recognize a specific object that appears into its view. The UART hardware connections are then made between the Pixy Camera and the SJOne board. The SJOne board registers are initialized and the baud rate is set to match the baud rate of the Pixy Camera. The task then continuously runs the read function in order to continuously grab data from the Pixy Camera. The SJOne board then continuously checks to see if the received data from the Pixy Camera matches the sync byte 0xaa55. This tells the SJOne board that the camera has picked up on an object. Once this byte is received the following bytes will include information regarding the recognized object. Below is the flow of the UART communication, as well as the Pseudo Code behind the implementation.

[[File: Pixy to SJOne board UART flowchart.png| frame | center| <center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>]]

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion, we were able to complete necessary for our project. All though we done what was needed, we believe that it isn't enough and we could've have done more. When we were testing we ran into problems like every other group, but we able to navigate past them successfully. We are glad we took this Embedded Systems class, because we able to learn how FREE RTOS operating system works and how we can implement it for our project. We have also learned how to implement different types of communication protocols and how to program their drivers. The main part we learned from this project is to make sure to set a team leader and make sure people follow their lead to finish the tasks. Within a group project, you have to hold your teammates accountable, make sure you set deadlines, make sure people are getting their work done and this is pretty important in real work place situations.

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.
*http://cmucam.org/projects/cmucam5/wiki/Pixy_Serial_Protocol
*http://cmucam.org/projects/cmucam5/wiki/Porting_Guide
*https://learn.sparkfun.com/tutorials/pulse-width-modulation
*http://www.socialledge.com/sjsu/index.php?title=Realtime_OS_on_Embedded_Systems

F16: Object Detector

2016-12-20T23:06:30Z

146 user5:

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center>]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: Pixy Camera.jpg| frame | center|<center>'''Figure 2. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File: Pixy Camera Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: Servo Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receive and sending bytes of information, and the run function that executes the byte transfer between the Pixy Camera and the SJOne board. Below is the Pseudocode an explanation for each function utilized within the UART driver.

<center><big> ''Initialization''</big> </center>

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below. From this equation we derive the DLL and DLM registers which are used as the divisors to establish the desired baud rate for the data transfer. The DLAB bit must also be initialized at is used as both the Receiver Buffer Register and the Transmit Holding Register. Both of which are set at the LCR register. The LCR register is also responsible for setting the start and stop bits of the UART communication.

[[File: Baud Rate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

''Initialization PseudoCode''

PCONP = Power on UART2
PCLKSEL1 = Set Peripheral Clock
PINSEL4 = Set RX and TX pin functionalities
// Acquire DLL_Value from desired Baud Rate
DLL_Value = clock/(16*Baud)
Set DLM and DLL registers according to DLL_Value
LCR = Set 8 bit data transfer Length
LCR = Set start and stop bit for data transfer

<center><big> ''Receiving Data''</big> </center>

Since the SJOne board is only receiving data from the Pixy Camera and not sending data in this situation only a receive byte function is required. The function waits to see if the data is transmitted from the Pixy Camera to the SJOne board by checking the RDR (Receiver Data Ready) register. Once the value of this register is set to 1, the receive function can then capture the transmitted data by looking at the RBR (Receiver Buffer Register).

''Receiving Function PseudoCode''

while (1){
if(RDR){
break;
}
}
new_data = RBR;

=== Pixy Camera Software Implementation ===

The Pixy Camera is able to externally communicate with micro-controllers by sending data regarding the object it is tracking, as well as receiving data for controlling the Pixy Camera itself. When the Pixy camera locates a recognized or preprogrammed object it begins to send information to a connected peripheral in bytes regarding that object. The first few bytes the Pixy camera sends are sync bytes, in other words bytes that let the external device know that the Pixy Camera has picked up on an object. The rest of the following bytes indicate the signature of the object, where it is on the screen with respect to the camera view (x/y center), as well as the width and the height of the object. Below are the specific bytes the Pixy Camera send to the SJOne board when it picks up on a device.

[[File: Pixy_ObjectBlock.png| frame | center| <center> '''Figure 7. Pixy Camera Object Block Data transfer''' </center>]]

The SJOne board is also able to control information to the Pixy Camera in order to control the pan and tilt of the servos, LED color, and camera brightness. However, the pan and tilt byte information is our area of concern with respect to control information being sent to the Pixy Camera. If the correct bytes are sent then the SJOne board acts as a master driving the pan and tilt servos of the Pixy Camera.

[[File: Pan & Tilt-Servo Control.png| frame | center| <center> '''Figure 8. Pan & Tilt Byte information''' </center>]]

=== Implementation ===

Initially, the Pixy Camera is programmed to recognize a specific object that appears into its view. The UART hardware connections are then made between the Pixy Camera and the SJOne board. The SJOne board registers are initialized and the baud rate is set to match the baud rate of the Pixy Camera. The task then continuously runs the read function in order to continuously grab data from the Pixy Camera. The SJOne board then continuously checks to see if the received data from the Pixy Camera matches the sync byte 0xaa55. This tells the SJOne board that the camera has picked up on an object. Once this byte is received the following bytes will include information regarding the recognized object. Below is the flow of the UART communication, as well as the Pseudo Code behind the implementation.

[[File: Pixy to SJOne board UART flowchart.png| frame | center| <center> '''Figure 9. Pixy to SJOne board UART flowchart''' </center>]]

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion, we were able to complete necessary for our project. All though we done what was needed, we believe that it isn't enough and we could've have done more. When we were testing we ran into problems like every other group, but we able to navigate past them successfully. We are glad we took this Embedded Systems class, because we able to learn how FREE RTOS operating system works and how we can implement it for our project. We have also learned how to implement different types of communication protocols and how to program their drivers. The main part we learned from this project is to make sure to set a team leader and make sure people follow their lead to finish the tasks. Within a group project, you have to hold your teammates accountable, make sure you set deadlines, make sure people are getting their work done and this is pretty important in real work place situations.

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.
*http://cmucam.org/projects/cmucam5/wiki/Pixy_Serial_Protocol
*http://cmucam.org/projects/cmucam5/wiki/Porting_Guide
*https://learn.sparkfun.com/tutorials/pulse-width-modulation
*http://www.socialledge.com/sjsu/index.php?title=Realtime_OS_on_Embedded_Systems

F16: Object Detector

2016-12-20T23:05:26Z

146 user5: /* Objectives & Introduction */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the UART Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop UART Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop UART Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center>]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: Pixy Camera.jpg| frame | center|<center>'''Figure 2. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File: Pixy Camera Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: Servo Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receive and sending bytes of information, and the run function that executes the byte transfer between the Pixy Camera and the SJOne board. Below is the Pseudocode an explanation for each function utilized within the UART driver.

<center><big> ''Initialization''</big> </center>

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below. From this equation we derive the DLL and DLM registers which are used as the divisors to establish the desired baud rate for the data transfer. The DLAB bit must also be initialized at is used as both the Receiver Buffer Register and the Transmit Holding Register. Both of which are set at the LCR register. The LCR register is also responsible for setting the start and stop bits of the UART communication.

[[File: Baud Rate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

''Initialization PseudoCode''

PCONP = Power on UART2
PCLKSEL1 = Set Peripheral Clock
PINSEL4 = Set RX and TX pin functionalities
// Acquire DLL_Value from desired Baud Rate
DLL_Value = clock/(16*Baud)
Set DLM and DLL registers according to DLL_Value
LCR = Set 8 bit data transfer Length
LCR = Set start and stop bit for data transfer

<center><big> ''Receiving Data''</big> </center>

Since the SJOne board is only receiving data from the Pixy Camera and not sending data in this situation only a receive byte function is required. The function waits to see if the data is transmitted from the Pixy Camera to the SJOne board by checking the RDR (Receiver Data Ready) register. Once the value of this register is set to 1, the receive function can then capture the transmitted data by looking at the RBR (Receiver Buffer Register).

''Receiving Function PseudoCode''

while (1){
if(RDR){
break;
}
}
new_data = RBR;

=== Pixy Camera Software Implementation ===

The Pixy Camera is able to externally communicate with micro-controllers by sending data regarding the object it is tracking, as well as receiving data for controlling the Pixy Camera itself. When the Pixy camera locates a recognized or preprogrammed object it begins to send information to a connected peripheral in bytes regarding that object. The first few bytes the Pixy camera sends are sync bytes, in other words bytes that let the external device know that the Pixy Camera has picked up on an object. The rest of the following bytes indicate the signature of the object, where it is on the screen with respect to the camera view (x/y center), as well as the width and the height of the object. Below are the specific bytes the Pixy Camera send to the SJOne board when it picks up on a device.

[[File: Pixy_ObjectBlock.png| frame | center| <center> '''Figure 7. Pixy Camera Object Block Data transfer''' </center>]]

The SJOne board is also able to control information to the Pixy Camera in order to control the pan and tilt of the servos, LED color, and camera brightness. However, the pan and tilt byte information is our area of concern with respect to control information being sent to the Pixy Camera. If the correct bytes are sent then the SJOne board acts as a master driving the pan and tilt servos of the Pixy Camera.

[[File: Pan & Tilt-Servo Control.png| frame | center| <center> '''Figure 8. Pan & Tilt Byte information''' </center>]]

=== Implementation ===

Initially, the Pixy Camera is programmed to recognize a specific object that appears into its view. The UART hardware connections are then made between the Pixy Camera and the SJOne board. The SJOne board registers are initialized and the baud rate is set to match the baud rate of the Pixy Camera. The task then continuously runs the read function in order to continuously grab data from the Pixy Camera. The SJOne board then continuously checks to see if the received data from the Pixy Camera matches the sync byte 0xaa55. This tells the SJOne board that the camera has picked up on an object. Once this byte is received the following bytes will include information regarding the recognized object. Below is the flow of the UART communication, as well as the Pseudo Code behind the implementation.

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion, we were able to complete necessary for our project. All though we done what was needed, we believe that it isn't enough and we could've have done more. When we were testing we ran into problems like every other group, but we able to navigate past them successfully. We are glad we took this Embedded Systems class, because we able to learn how FREE RTOS operating system works and how we can implement it for our project. We have also learned how to implement different types of communication protocols and how to program their drivers. The main part we learned from this project is to make sure to set a team leader and make sure people follow their lead to finish the tasks. Within a group project, you have to hold your teammates accountable, make sure you set deadlines, make sure people are getting their work done and this is pretty important in real work place situations.

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.
*http://cmucam.org/projects/cmucam5/wiki/Pixy_Serial_Protocol
*http://cmucam.org/projects/cmucam5/wiki/Porting_Guide
*https://learn.sparkfun.com/tutorials/pulse-width-modulation
*http://www.socialledge.com/sjsu/index.php?title=Realtime_OS_on_Embedded_Systems

F16: Object Detector

2016-12-20T22:58:59Z

146 user5: /* Implementation */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center>]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: Pixy Camera.jpg| frame | center|<center>'''Figure 2. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File: Pixy Camera Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: Servo Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receive and sending bytes of information, and the run function that executes the byte transfer between the Pixy Camera and the SJOne board. Below is the Pseudocode an explanation for each function utilized within the UART driver.

<center><big> ''Initialization''</big> </center>

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below. From this equation we derive the DLL and DLM registers which are used as the divisors to establish the desired baud rate for the data transfer. The DLAB bit must also be initialized at is used as both the Receiver Buffer Register and the Transmit Holding Register. Both of which are set at the LCR register. The LCR register is also responsible for setting the start and stop bits of the UART communication.

[[File: Baud Rate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

''Initialization PseudoCode''

PCONP = Power on UART2
PCLKSEL1 = Set Peripheral Clock
PINSEL4 = Set RX and TX pin functionalities
// Acquire DLL_Value from desired Baud Rate
DLL_Value = clock/(16*Baud)
Set DLM and DLL registers according to DLL_Value
LCR = Set 8 bit data transfer Length
LCR = Set start and stop bit for data transfer

<center><big> ''Receiving Data''</big> </center>

Since the SJOne board is only receiving data from the Pixy Camera and not sending data in this situation only a receive byte function is required. The function waits to see if the data is transmitted from the Pixy Camera to the SJOne board by checking the RDR (Receiver Data Ready) register. Once the value of this register is set to 1, the receive function can then capture the transmitted data by looking at the RBR (Receiver Buffer Register).

''Receiving Function PseudoCode''

while (1){
if(RDR){
break;
}
}
new_data = RBR;

=== Pixy Camera Software Implementation ===

The Pixy Camera is able to externally communicate with micro-controllers by sending data regarding the object it is tracking, as well as receiving data for controlling the Pixy Camera itself. When the Pixy camera locates a recognized or preprogrammed object it begins to send information to a connected peripheral in bytes regarding that object. The first few bytes the Pixy camera sends are sync bytes, in other words bytes that let the external device know that the Pixy Camera has picked up on an object. The rest of the following bytes indicate the signature of the object, where it is on the screen with respect to the camera view (x/y center), as well as the width and the height of the object. Below are the specific bytes the Pixy Camera send to the SJOne board when it picks up on a device.

[[File: Pixy_ObjectBlock.png| frame | center| <center> '''Figure 7. Pixy Camera Object Block Data transfer''' </center>]]

The SJOne board is also able to control information to the Pixy Camera in order to control the pan and tilt of the servos, LED color, and camera brightness. However, the pan and tilt byte information is our area of concern with respect to control information being sent to the Pixy Camera. If the correct bytes are sent then the SJOne board acts as a master driving the pan and tilt servos of the Pixy Camera.

[[File: Pan & Tilt-Servo Control.png| frame | center| <center> '''Figure 8. Pan & Tilt Byte information''' </center>]]

=== Implementation ===

Initially, the Pixy Camera is programmed to recognize a specific object that appears into its view. The UART hardware connections are then made between the Pixy Camera and the SJOne board. The SJOne board registers are initialized and the baud rate is set to match the baud rate of the Pixy Camera. The task then continuously runs the read function in order to continuously grab data from the Pixy Camera. The SJOne board then continuously checks to see if the received data from the Pixy Camera matches the sync byte 0xaa55. This tells the SJOne board that the camera has picked up on an object. Once this byte is received the following bytes will include information regarding the recognized object. Below is the flow of the UART communication, as well as the Pseudo Code behind the implementation.

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion, we were able to complete necessary for our project. All though we done what was needed, we believe that it isn't enough and we could've have done more. When we were testing we ran into problems like every other group, but we able to navigate past them successfully. We are glad we took this Embedded Systems class, because we able to learn how FREE RTOS operating system works and how we can implement it for our project. We have also learned how to implement different types of communication protocols and how to program their drivers. The main part we learned from this project is to make sure to set a team leader and make sure people follow their lead to finish the tasks. Within a group project, you have to hold your teammates accountable, make sure you set deadlines, make sure people are getting their work done and this is pretty important in real work place situations.

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.
*http://cmucam.org/projects/cmucam5/wiki/Pixy_Serial_Protocol
*http://cmucam.org/projects/cmucam5/wiki/Porting_Guide
*https://learn.sparkfun.com/tutorials/pulse-width-modulation
*http://www.socialledge.com/sjsu/index.php?title=Realtime_OS_on_Embedded_Systems

F16: Object Detector

2016-12-20T22:27:18Z

146 user5: /* Conclusion */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center>]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: Pixy Camera.jpg| frame | center|<center>'''Figure 2. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File: Pixy Camera Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: Servo Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receive and sending bytes of information, and the run function that executes the byte transfer between the Pixy Camera and the SJOne board. Below is the Pseudocode an explanation for each function utilized within the UART driver.

<center><big> ''Initialization''</big> </center>

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below. From this equation we derive the DLL and DLM registers which are used as the divisors to establish the desired baud rate for the data transfer. The DLAB bit must also be initialized at is used as both the Receiver Buffer Register and the Transmit Holding Register. Both of which are set at the LCR register. The LCR register is also responsible for setting the start and stop bits of the UART communication.

[[File: Baud Rate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

''Initialization PseudoCode''

PCONP = Power on UART2
PCLKSEL1 = Set Peripheral Clock
PINSEL4 = Set RX and TX pin functionalities
// Acquire DLL_Value from desired Baud Rate
DLL_Value = clock/(16*Baud)
Set DLM and DLL registers according to DLL_Value
LCR = Set 8 bit data transfer Length
LCR = Set start and stop bit for data transfer

<center><big> ''Receiving Data''</big> </center>

Since the SJOne board is only receiving data from the Pixy Camera and not sending data in this situation only a receive byte function is required. The function waits to see if the data is transmitted from the Pixy Camera to the SJOne board by checking the RDR (Receiver Data Ready) register. Once the value of this register is set to 1, the receive function can then capture the transmitted data by looking at the RBR (Receiver Buffer Register).

''Receiving Function PseudoCode''

while (1){
if(RDR){
break;
}
}
new_data = RBR;

=== Pixy Camera Software Implementation ===

The Pixy Camera is able to externally communicate with micro-controllers by sending data regarding the object it is tracking, as well as receiving data for controlling the Pixy Camera itself. When the Pixy camera locates a recognized or preprogrammed object it begins to send information to a connected peripheral in bytes regarding that object. The first few bytes the Pixy camera sends are sync bytes, in other words bytes that let the external device know that the Pixy Camera has picked up on an object. The rest of the following bytes indicate the signature of the object, where it is on the screen with respect to the camera view (x/y center), as well as the width and the height of the object. Below are the specific bytes the Pixy Camera send to the SJOne board when it picks up on a device.

[[File: Pixy_ObjectBlock.png| frame | center| <center> '''Figure 7. Pixy Camera Object Block Data transfer''' </center>]]

The SJOne board is also able to control information to the Pixy Camera in order to control the pan and tilt of the servos, LED color, and camera brightness. However, the pan and tilt byte information is our area of concern with respect to control information being sent to the Pixy Camera. If the correct bytes are sent then the SJOne board acts as a master driving the pan and tilt servos of the Pixy Camera.

[[File: Pan & Tilt-Servo Control.png| frame | center| <center> '''Figure 8. Pan & Tilt Byte information''' </center>]]

=== Implementation ===
This section includes implementation, but again, not the details, just the high level. For example, you can list the steps it takes to communicate over a sensor, or the steps needed to write a page of memory onto SPI Flash. You can include sub-sections for each of your component implementation.

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion, we were able to complete necessary for our project. All though we done what was needed, we believe that it isn't enough and we could've have done more. When we were testing we ran into problems like every other group, but we able to navigate past them successfully. We are glad we took this Embedded Systems class, because we able to learn how FREE RTOS operating system works and how we can implement it for our project. We have also learned how to implement different types of communication protocols and how to program their drivers. The main part we learned from this project is to make sure to set a team leader and make sure people follow their lead to finish the tasks. Within a group project, you have to hold your teammates accountable, make sure you set deadlines, make sure people are getting their work done and this is pretty important in real work place situations.

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.
*http://cmucam.org/projects/cmucam5/wiki/Pixy_Serial_Protocol
*http://cmucam.org/projects/cmucam5/wiki/Porting_Guide
*https://learn.sparkfun.com/tutorials/pulse-width-modulation
*http://www.socialledge.com/sjsu/index.php?title=Realtime_OS_on_Embedded_Systems

F16: Object Detector

2016-12-20T22:25:05Z

146 user5:

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center>]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: Pixy Camera.jpg| frame | center|<center>'''Figure 2. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File: Pixy Camera Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: Servo Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receive and sending bytes of information, and the run function that executes the byte transfer between the Pixy Camera and the SJOne board. Below is the Pseudocode an explanation for each function utilized within the UART driver.

<center><big> ''Initialization''</big> </center>

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below. From this equation we derive the DLL and DLM registers which are used as the divisors to establish the desired baud rate for the data transfer. The DLAB bit must also be initialized at is used as both the Receiver Buffer Register and the Transmit Holding Register. Both of which are set at the LCR register. The LCR register is also responsible for setting the start and stop bits of the UART communication.

[[File: Baud Rate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

''Initialization PseudoCode''

PCONP = Power on UART2
PCLKSEL1 = Set Peripheral Clock
PINSEL4 = Set RX and TX pin functionalities
// Acquire DLL_Value from desired Baud Rate
DLL_Value = clock/(16*Baud)
Set DLM and DLL registers according to DLL_Value
LCR = Set 8 bit data transfer Length
LCR = Set start and stop bit for data transfer

<center><big> ''Receiving Data''</big> </center>

Since the SJOne board is only receiving data from the Pixy Camera and not sending data in this situation only a receive byte function is required. The function waits to see if the data is transmitted from the Pixy Camera to the SJOne board by checking the RDR (Receiver Data Ready) register. Once the value of this register is set to 1, the receive function can then capture the transmitted data by looking at the RBR (Receiver Buffer Register).

''Receiving Function PseudoCode''

while (1){
if(RDR){
break;
}
}
new_data = RBR;

=== Pixy Camera Software Implementation ===

The Pixy Camera is able to externally communicate with micro-controllers by sending data regarding the object it is tracking, as well as receiving data for controlling the Pixy Camera itself. When the Pixy camera locates a recognized or preprogrammed object it begins to send information to a connected peripheral in bytes regarding that object. The first few bytes the Pixy camera sends are sync bytes, in other words bytes that let the external device know that the Pixy Camera has picked up on an object. The rest of the following bytes indicate the signature of the object, where it is on the screen with respect to the camera view (x/y center), as well as the width and the height of the object. Below are the specific bytes the Pixy Camera send to the SJOne board when it picks up on a device.

[[File: Pixy_ObjectBlock.png| frame | center| <center> '''Figure 7. Pixy Camera Object Block Data transfer''' </center>]]

The SJOne board is also able to control information to the Pixy Camera in order to control the pan and tilt of the servos, LED color, and camera brightness. However, the pan and tilt byte information is our area of concern with respect to control information being sent to the Pixy Camera. If the correct bytes are sent then the SJOne board acts as a master driving the pan and tilt servos of the Pixy Camera.

[[File: Pan & Tilt-Servo Control.png| frame | center| <center> '''Figure 8. Pan & Tilt Byte information''' </center>]]

=== Implementation ===
This section includes implementation, but again, not the details, just the high level. For example, you can list the steps it takes to communicate over a sensor, or the steps needed to write a page of memory onto SPI Flash. You can include sub-sections for each of your component implementation.

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion,

Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.
*http://cmucam.org/projects/cmucam5/wiki/Pixy_Serial_Protocol
*http://cmucam.org/projects/cmucam5/wiki/Porting_Guide
*https://learn.sparkfun.com/tutorials/pulse-width-modulation
*http://www.socialledge.com/sjsu/index.php?title=Realtime_OS_on_Embedded_Systems

Realtime OS on Embedded Systems

2016-12-20T22:06:05Z

146 user5: /* Fall 2016 */

== Program History ==
My contribution in Embedded System courses started with CmpE146. This course teaches students on how to write UART, SPI, and I2C Drivers and interface their drivers to peripherals. There are about 8 weekly labs in which students not only write drivers, but also learn the core of Real-time Operating Systems including threading (tasks), and inter-task communication using Queues and Semaphores. FreeRTOS is the operating system used with C/C++ Compiler based on GNU.

When the course was started by Dr. Ozemek @ SJSU (sometime before 2005), not many resources were out there. Still, with helpful guidance from Dr. Ozemek, interesting projects were created. This is when I started to help out in Embedded System Courses, and by collecting and sharing knowledge, we've raised the bar at SJSU!

There have been many great projects at SJSU, but since no one knew about them, the hard work went to a waste for anyone but the creator. But now we've got the infrastructure to share the projects, which turn out as great references for future students. Here is my project that started around 2007, and turned into Bachelor's Senior Design Project: 
[http://www.youtube.com/watch?v=QEadXdRl3ws&feature=plcp YouTube Video of Self-Navigating Car]

As of 2013, I have broadened my contribution to other embedded system courses like CmpE240, CmpE243 and CmpE244.

== Lab Assignments ==
This article contains laboratory assignments and resources. The assignments are under construction as we move towards SJ-One development board.
* [[Embedded System Tutorial GPIO | Lesson 1 : GPIO]]
* [[Embedded System Tutorial UART | Lesson 2 : UART]]
* [[Embedded System Tutorial SPI | Lesson 3 : SPI]]
* [[Embedded System I2C Tutorial | Lesson 4 : I2C]]
* [[Embedded System Tutorial Interrupts | Lesson 5 : Interrupts]]
* [[Embedded System Tutorial FreeRTOS | Lesson 6 : FreeRTOS Tasks]]
* [[Embedded System Tutorial File I/O | Lesson 7 : FreeRTOS Application Programming]]

==Other reference articles==
* [[Bitmasking Tutorial]] (+ GPIO Example)
* [[ LPC17xx Memory Map & Interrupts]]

 
== Senior Design Projects ==

 
== Semester Projects ==
Every semester, students are given about 7-10 weeks to complete their projects. During this short time-span, students form groups, order parts, and begin working on their projects. The work performed during the semester is documented at this Wiki.

Here is the list of Preet's documented projects: 
* [[Preet's Relay Controller Project]]
* [[Nordic Low Powered Mesh Network stack]]
* [http://www.youtube.com/watch?v=QEadXdRl3ws&feature=plcp Senior Design Project (MS-CmpE) Video]

Here is another resource for good project references :
[http://people.ece.cornell.edu/land/courses/ece4760/FinalProjects/ Cornell EE476 Projects]

 
<HR>
=== Appendix ===
=== [[Fall 2016 | Fall 2016]] ===

CMPE146:
* Add Your Group on Here, then follow the link to add more to your template *
* [http://www.socialledge.com/sjsu/index.php?title=F16:_Seismograph F16: Seismograph]
* [[F16: Piano Glove]]
* [[F16: Object Detector]]
* [[F16: Autonomous Nautical System]]
* [[F16: Autonomous Fire Extinguishing Vehicle]]
* [[F16: Autonomous Runaway Alarm Car]]
* [[F16: E-Bike]]
* [[F16: NotifyBox]]
* [[F16: Wireless Tilt Controlled Camera Arm]]
* [[F16: OBD2 Reader]]
* [[F16: Micro Watch Monitoring System]]
* [[F16: Door Alarm System]]
* [[F16: I2Coffee]]
* [[F16: Real Time Traffic Control System (RTTCS)]]

=== [[Spring 2016 | Spring 2016]] ===
* [[S16: Fantastic Four]]
* [[S16: Simpsons]]
* [[S16: Mars 1]]
* [[S16: OpenSJ Bluz]]
* [[S16: Motion Copy Bot]]
* [[S16: Biker Assist]]
* [[S16: Helios]]
* [[S16: Sound Buddy]]
* [[S16: Warriors]]
* [[S16: Expendables]]
* [[S16: Ahava]]
* [[S16: Number 1]]
* [[S16: SkyNet]]
* [[S16: SmartDoorLock]]

Cmpe 146:
* [[S16: Camera Gimbal]]
* [[S16: Laser Harp]]
* <strike>[[S16: Laser Cutter Motor Controller]]</strike>
* [[S16: Sprinkler]]
* [[S16: The Jatrick Car]]
* [[S16: Dan]]
* [[S16: Robolamp]]
* [[S16: Pinball]]
 
<HR>

=== [[Fall 2015 | Fall 2015]] ===

CmpE146:
* [[F15: Autonomous Mobile]]
* [[F15: Car Report]]
* [[F15: Electronic Piano]]
* [[F15: Doorknock over Bluetooth]]
* [[F15: Smart Car]]
* [[F15: Plant Control]]
* [[F15: Laser Security System]]

 
<HR>

=== [[CmpE244 Spring 2015 | Spring 2015]] ===

 
* [[S15: Quadcopter - It flies]]
* [[S15: Remote Learner]]
* [[S15: Protocol Interface: I2C - CAN Bridge]]
* [[S15: Vision RC Car]]
* [[S15: SJeight Octocopter]]
* [[S15: Swarm Robots]]
* [[S15: Smart Sparta Parking System]]
* [[S15: Touch Navigator]]
* [[S15: Wizard's Chess System]]
* [[S15: Bug Rider]]
* [[S15: Real Time Brake Assist (RTBA)]]
* [[S15: Wireless Mesh Network]]
* [[S15: Wireless Power Transfer System]]
* [[S15: Drone]]
* [[S15: Tree Node using Google Protocol Buffers]]
* [[S15: Multi-media Car]]
* [[S15: Hand Gesture Recognition using IR Sensors]]
* [[S15: CAN controlled RGB LED cubes]]
* [[S15: Rubik's Cube Solver]]
* [[S15: RFID Security Box]]
* [[S15: Automated Meeting Room Reservation]]
* [[S15: Patient Buddy System (PBS)]]

CmpE146:
* [[S15: Hovercopter]]
* [[S15: Triclops: Smart RC Car]]
* [[S15: Connect Four - Robotic Player]]
* [[S15: Self-Balancing Robot]]
* [[S15: MP3 Player with Graphic Equalizer Display]]
* [[S15: Motion-Controlled RC Car]]
* [[S15: MENL (Monster Encounter Night Light) ]]
* [[S15: Tilt Motion Controlled LED Alarm Clock]]
* [[S15: Alarm Based Coffee Maker]]

=== [[CmpE244 Spring 2014 | Spring 2014]] ===

 
* Senior Project: [[Project Advising: Remote Security System]]

 
* [[S14: Quadcopter]]
* [[S14: Smart Weather Clock]]
* [[S14: Divine WINd]]
* [[S14: Data Acquisition using CAN bus]]
* [[S14: E-Ink Display for Shopping Tags]]
* [[S14: Spectrum Analyzer for Audio Frequency Signals]]
* [[S14: CAN Firmware Uploader]]
* [[S14: Asset Management and Location System]]
* [[S14: Location Tracker]]
* [[S14: Androbot]]
* [[S14: Virtual Dog]]
* [[S14: Android based Automation]]
* [[S14: FaceTime Robo]]
* [[S14: Wireless Control Car]]
* [[S14: Power Efficient Security Door System for Light-rail using CAN Bus]]
* [[S14: Android based home monitoring system]]
* [[S14: Need For Speed]]

CmpE146
* [[S14: Hyperintelligent NFC Locker of the Future]]
* [[S14: Smart Planter]]
* [[S14: Modular Security System]]
* [[S14: Autonomous Control System]]
* [[S14: Anti-Crash Car]]
* [[S14: Tricopter]]

=== [[CmpE240 Fall 2013 | Fall 2013]] ===

* [[F13: POV Display]]
* [[F13: Line Following Robot]]
* [[F13: LED Display]]
* [[F13: Bulb Ramper]]
* [[F13: Garage Parking Assistant]]
* [[F13: Quadcopter]]
* [[F13: BarkMaster2000]]
* [[F13: Remote Control Car]]
* [[F13: Obstacle Avoidance Robot]]
* [[F13: Vehicle On Board Diagnostics]]

=== [[CmpE146 Spring 2013 | Spring 2013]] ===

* [[S13: 2D Plotter]]
* [[S13: Smart Cube]]
* [[S13: Garage Parking Aid]]
* [[S13: Smart Security]]
* [[S13: Door Alarm System]]
* [[S13: Solar Panel Tracker]]

=== [[CmpE146 Fall 2012|Fall 2012]] ===

* [[F12: Evil Watchdog]]
* [[F12: Smart Bulb]]
* [[F12: All Your Base are Belong to You]]
* [[F12: Android Controlled MP3]]
* [[F12: Unified Wireless Health Monitoring System]]
* [[F12: OBD-II Android Monitor]]
* [[F12: Self-Driving GPS Following Car]]
* [[F12: Android Door Lock]]

=== [[CmpE146 Spring 2012|Spring 2012]] ===
* [[S12: FreeRTOS based QuadCopter]]
* [[S12: Web-based MP3 Player]]
* [[S12: Self Drive Car]]
* [[S12: VAndroid]]
* [[S12: Traffic Light Sensing Vehicle]]
* [[S12: Sound Reader]]
* [[S12: Remote Controlled MP3 Player]]
* [[S12: Android Controlled Robot]]
* [[S12: Eyes-Free GPS]]

 

== Handy References ==
* [[Sample Project Report]]
* [[Project Proposal Guidelines]]
* [[CmpE146 Lab. Resources]]

Realtime OS on Embedded Systems

2016-12-20T22:05:33Z

146 user5: /* Fall 2016 */

== Program History ==
My contribution in Embedded System courses started with CmpE146. This course teaches students on how to write UART, SPI, and I2C Drivers and interface their drivers to peripherals. There are about 8 weekly labs in which students not only write drivers, but also learn the core of Real-time Operating Systems including threading (tasks), and inter-task communication using Queues and Semaphores. FreeRTOS is the operating system used with C/C++ Compiler based on GNU.

When the course was started by Dr. Ozemek @ SJSU (sometime before 2005), not many resources were out there. Still, with helpful guidance from Dr. Ozemek, interesting projects were created. This is when I started to help out in Embedded System Courses, and by collecting and sharing knowledge, we've raised the bar at SJSU!

There have been many great projects at SJSU, but since no one knew about them, the hard work went to a waste for anyone but the creator. But now we've got the infrastructure to share the projects, which turn out as great references for future students. Here is my project that started around 2007, and turned into Bachelor's Senior Design Project: 
[http://www.youtube.com/watch?v=QEadXdRl3ws&feature=plcp YouTube Video of Self-Navigating Car]

As of 2013, I have broadened my contribution to other embedded system courses like CmpE240, CmpE243 and CmpE244.

== Lab Assignments ==
This article contains laboratory assignments and resources. The assignments are under construction as we move towards SJ-One development board.
* [[Embedded System Tutorial GPIO | Lesson 1 : GPIO]]
* [[Embedded System Tutorial UART | Lesson 2 : UART]]
* [[Embedded System Tutorial SPI | Lesson 3 : SPI]]
* [[Embedded System I2C Tutorial | Lesson 4 : I2C]]
* [[Embedded System Tutorial Interrupts | Lesson 5 : Interrupts]]
* [[Embedded System Tutorial FreeRTOS | Lesson 6 : FreeRTOS Tasks]]
* [[Embedded System Tutorial File I/O | Lesson 7 : FreeRTOS Application Programming]]

==Other reference articles==
* [[Bitmasking Tutorial]] (+ GPIO Example)
* [[ LPC17xx Memory Map & Interrupts]]

 
== Senior Design Projects ==

 
== Semester Projects ==
Every semester, students are given about 7-10 weeks to complete their projects. During this short time-span, students form groups, order parts, and begin working on their projects. The work performed during the semester is documented at this Wiki.

Here is the list of Preet's documented projects: 
* [[Preet's Relay Controller Project]]
* [[Nordic Low Powered Mesh Network stack]]
* [http://www.youtube.com/watch?v=QEadXdRl3ws&feature=plcp Senior Design Project (MS-CmpE) Video]

Here is another resource for good project references :
[http://people.ece.cornell.edu/land/courses/ece4760/FinalProjects/ Cornell EE476 Projects]

 
<HR>
=== Appendix ===
=== [[Fall 2016 | Fall 2016]] ===

CMPE146:
* Add Your Group on Here, then follow the link to add more to your template *
* [http://www.socialledge.com/sjsu/index.php?title=F16:_Seismograph F16: Seismograph]
* [[F16: Piano Glove]]
* [[F16: Object Detector]]
* [[F16: Pattern Detector]]
* [[F16: Autonomous Nautical System]]
* [[F16: Autonomous Fire Extinguishing Vehicle]]
* [[F16: Autonomous Runaway Alarm Car]]
* [[F16: E-Bike]]
* [[F16: NotifyBox]]
* [[F16: Wireless Tilt Controlled Camera Arm]]
* [[F16: OBD2 Reader]]
* [[F16: Micro Watch Monitoring System]]
* [[F16: Door Alarm System]]
* [[F16: I2Coffee]]
* [[F16: Real Time Traffic Control System (RTTCS)]]

=== [[Spring 2016 | Spring 2016]] ===
* [[S16: Fantastic Four]]
* [[S16: Simpsons]]
* [[S16: Mars 1]]
* [[S16: OpenSJ Bluz]]
* [[S16: Motion Copy Bot]]
* [[S16: Biker Assist]]
* [[S16: Helios]]
* [[S16: Sound Buddy]]
* [[S16: Warriors]]
* [[S16: Expendables]]
* [[S16: Ahava]]
* [[S16: Number 1]]
* [[S16: SkyNet]]
* [[S16: SmartDoorLock]]

Cmpe 146:
* [[S16: Camera Gimbal]]
* [[S16: Laser Harp]]
* <strike>[[S16: Laser Cutter Motor Controller]]</strike>
* [[S16: Sprinkler]]
* [[S16: The Jatrick Car]]
* [[S16: Dan]]
* [[S16: Robolamp]]
* [[S16: Pinball]]
 
<HR>

=== [[Fall 2015 | Fall 2015]] ===

CmpE146:
* [[F15: Autonomous Mobile]]
* [[F15: Car Report]]
* [[F15: Electronic Piano]]
* [[F15: Doorknock over Bluetooth]]
* [[F15: Smart Car]]
* [[F15: Plant Control]]
* [[F15: Laser Security System]]

 
<HR>

=== [[CmpE244 Spring 2015 | Spring 2015]] ===

 
* [[S15: Quadcopter - It flies]]
* [[S15: Remote Learner]]
* [[S15: Protocol Interface: I2C - CAN Bridge]]
* [[S15: Vision RC Car]]
* [[S15: SJeight Octocopter]]
* [[S15: Swarm Robots]]
* [[S15: Smart Sparta Parking System]]
* [[S15: Touch Navigator]]
* [[S15: Wizard's Chess System]]
* [[S15: Bug Rider]]
* [[S15: Real Time Brake Assist (RTBA)]]
* [[S15: Wireless Mesh Network]]
* [[S15: Wireless Power Transfer System]]
* [[S15: Drone]]
* [[S15: Tree Node using Google Protocol Buffers]]
* [[S15: Multi-media Car]]
* [[S15: Hand Gesture Recognition using IR Sensors]]
* [[S15: CAN controlled RGB LED cubes]]
* [[S15: Rubik's Cube Solver]]
* [[S15: RFID Security Box]]
* [[S15: Automated Meeting Room Reservation]]
* [[S15: Patient Buddy System (PBS)]]

CmpE146:
* [[S15: Hovercopter]]
* [[S15: Triclops: Smart RC Car]]
* [[S15: Connect Four - Robotic Player]]
* [[S15: Self-Balancing Robot]]
* [[S15: MP3 Player with Graphic Equalizer Display]]
* [[S15: Motion-Controlled RC Car]]
* [[S15: MENL (Monster Encounter Night Light) ]]
* [[S15: Tilt Motion Controlled LED Alarm Clock]]
* [[S15: Alarm Based Coffee Maker]]

=== [[CmpE244 Spring 2014 | Spring 2014]] ===

 
* Senior Project: [[Project Advising: Remote Security System]]

 
* [[S14: Quadcopter]]
* [[S14: Smart Weather Clock]]
* [[S14: Divine WINd]]
* [[S14: Data Acquisition using CAN bus]]
* [[S14: E-Ink Display for Shopping Tags]]
* [[S14: Spectrum Analyzer for Audio Frequency Signals]]
* [[S14: CAN Firmware Uploader]]
* [[S14: Asset Management and Location System]]
* [[S14: Location Tracker]]
* [[S14: Androbot]]
* [[S14: Virtual Dog]]
* [[S14: Android based Automation]]
* [[S14: FaceTime Robo]]
* [[S14: Wireless Control Car]]
* [[S14: Power Efficient Security Door System for Light-rail using CAN Bus]]
* [[S14: Android based home monitoring system]]
* [[S14: Need For Speed]]

CmpE146
* [[S14: Hyperintelligent NFC Locker of the Future]]
* [[S14: Smart Planter]]
* [[S14: Modular Security System]]
* [[S14: Autonomous Control System]]
* [[S14: Anti-Crash Car]]
* [[S14: Tricopter]]

=== [[CmpE240 Fall 2013 | Fall 2013]] ===

* [[F13: POV Display]]
* [[F13: Line Following Robot]]
* [[F13: LED Display]]
* [[F13: Bulb Ramper]]
* [[F13: Garage Parking Assistant]]
* [[F13: Quadcopter]]
* [[F13: BarkMaster2000]]
* [[F13: Remote Control Car]]
* [[F13: Obstacle Avoidance Robot]]
* [[F13: Vehicle On Board Diagnostics]]

=== [[CmpE146 Spring 2013 | Spring 2013]] ===

* [[S13: 2D Plotter]]
* [[S13: Smart Cube]]
* [[S13: Garage Parking Aid]]
* [[S13: Smart Security]]
* [[S13: Door Alarm System]]
* [[S13: Solar Panel Tracker]]

=== [[CmpE146 Fall 2012|Fall 2012]] ===

* [[F12: Evil Watchdog]]
* [[F12: Smart Bulb]]
* [[F12: All Your Base are Belong to You]]
* [[F12: Android Controlled MP3]]
* [[F12: Unified Wireless Health Monitoring System]]
* [[F12: OBD-II Android Monitor]]
* [[F12: Self-Driving GPS Following Car]]
* [[F12: Android Door Lock]]

=== [[CmpE146 Spring 2012|Spring 2012]] ===
* [[S12: FreeRTOS based QuadCopter]]
* [[S12: Web-based MP3 Player]]
* [[S12: Self Drive Car]]
* [[S12: VAndroid]]
* [[S12: Traffic Light Sensing Vehicle]]
* [[S12: Sound Reader]]
* [[S12: Remote Controlled MP3 Player]]
* [[S12: Android Controlled Robot]]
* [[S12: Eyes-Free GPS]]

 

== Handy References ==
* [[Sample Project Report]]
* [[Project Proposal Guidelines]]
* [[CmpE146 Lab. Resources]]

F16: Object Detector

2016-12-20T22:04:10Z

146 user5: Created page with "== Abstract == Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will..."

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center>]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: Pixy Camera.jpg| frame | center|<center>'''Figure 2. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File: Pixy Camera Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: Servo Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receive and sending bytes of information, and the run function that executes the byte transfer between the Pixy Camera and the SJOne board. Below is the Pseudocode an explanation for each function utilized within the UART driver.

<center><big> ''Initialization''</big> </center>

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below. From this equation we derive the DLL and DLM registers which are used as the divisors to establish the desired baud rate for the data transfer. The DLAB bit must also be initialized at is used as both the Receiver Buffer Register and the Transmit Holding Register. Both of which are set at the LCR register. The LCR register is also responsible for setting the start and stop bits of the UART communication.

[[File: Baud Rate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

''Initialization PseudoCode''

PCONP = Power on UART2
PCLKSEL1 = Set Peripheral Clock
PINSEL4 = Set RX and TX pin functionalities
// Acquire DLL_Value from desired Baud Rate
DLL_Value = clock/(16*Baud)
Set DLM and DLL registers according to DLL_Value
LCR = Set 8 bit data transfer Length
LCR = Set start and stop bit for data transfer

<center><big> ''Receiving Data''</big> </center>

Since the SJOne board is only receiving data from the Pixy Camera and not sending data in this situation only a receive byte function is required. The function waits to see if the data is transmitted from the Pixy Camera to the SJOne board by checking the RDR (Receiver Data Ready) register. Once the value of this register is set to 1, the receive function can then capture the transmitted data by looking at the RBR (Receiver Buffer Register).

''Receiving Function PseudoCode''

while (1){
if(RDR){
break;
}
}
new_data = RBR;

=== Pixy Camera Software Implementation ===

The Pixy Camera is able to externally communicate with micro-controllers by sending data regarding the object it is tracking, as well as receiving data for controlling the Pixy Camera itself. When the Pixy camera locates a recognized or preprogrammed object it begins to send information to a connected peripheral in bytes regarding that object. The first few bytes the Pixy camera sends are sync bytes, in other words bytes that let the external device know that the Pixy Camera has picked up on an object. The rest of the following bytes indicate the signature of the object, where it is on the screen with respect to the camera view (x/y center), as well as the width and the height of the object. Below are the specific bytes the Pixy Camera send to the SJOne board when it picks up on a device.

[[File: Pixy_ObjectBlock.png| frame | center| <center> '''Figure 7. Pixy Camera Object Block Data transfer''' </center>]]

=== Implementation ===
This section includes implementation, but again, not the details, just the high level. For example, you can list the steps it takes to communicate over a sensor, or the steps needed to write a page of memory onto SPI Flash. You can include sub-sections for each of your component implementation.

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion,

Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.
*http://cmucam.org/projects/cmucam5/wiki/Pixy_Serial_Protocol
*http://cmucam.org/projects/cmucam5/wiki/Porting_Guide
*https://learn.sparkfun.com/tutorials/pulse-width-modulation
*http://www.socialledge.com/sjsu/index.php?title=Realtime_OS_on_Embedded_Systems

Realtime OS on Embedded Systems

2016-12-20T22:03:55Z

146 user5: /* Fall 2016 */

F16: Pattern Detector

2016-12-20T22:02:41Z

146 user5: /* References Used */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center>]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: Pixy Camera.jpg| frame | center|<center>'''Figure 2. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File: Pixy Camera Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: Servo Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receive and sending bytes of information, and the run function that executes the byte transfer between the Pixy Camera and the SJOne board. Below is the Pseudocode an explanation for each function utilized within the UART driver.

<center><big> ''Initialization''</big> </center>

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below. From this equation we derive the DLL and DLM registers which are used as the divisors to establish the desired baud rate for the data transfer. The DLAB bit must also be initialized at is used as both the Receiver Buffer Register and the Transmit Holding Register. Both of which are set at the LCR register. The LCR register is also responsible for setting the start and stop bits of the UART communication.

[[File: Baud Rate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

''Initialization PseudoCode''

PCONP = Power on UART2
PCLKSEL1 = Set Peripheral Clock
PINSEL4 = Set RX and TX pin functionalities
// Acquire DLL_Value from desired Baud Rate
DLL_Value = clock/(16*Baud)
Set DLM and DLL registers according to DLL_Value
LCR = Set 8 bit data transfer Length
LCR = Set start and stop bit for data transfer

<center><big> ''Receiving Data''</big> </center>

Since the SJOne board is only receiving data from the Pixy Camera and not sending data in this situation only a receive byte function is required. The function waits to see if the data is transmitted from the Pixy Camera to the SJOne board by checking the RDR (Receiver Data Ready) register. Once the value of this register is set to 1, the receive function can then capture the transmitted data by looking at the RBR (Receiver Buffer Register).

''Receiving Function PseudoCode''

while (1){
if(RDR){
break;
}
}
new_data = RBR;

=== Pixy Camera Software Implementation ===

The Pixy Camera is able to externally communicate with micro-controllers by sending data regarding the object it is tracking, as well as receiving data for controlling the Pixy Camera itself. When the Pixy camera locates a recognized or preprogrammed object it begins to send information to a connected peripheral in bytes regarding that object. The first few bytes the Pixy camera sends are sync bytes, in other words bytes that let the external device know that the Pixy Camera has picked up on an object. The rest of the following bytes indicate the signature of the object, where it is on the screen with respect to the camera view (x/y center), as well as the width and the height of the object. Below are the specific bytes the Pixy Camera send to the SJOne board when it picks up on a device.

[[File: Pixy_ObjectBlock.png| frame | center| <center> '''Figure 7. Pixy Camera Object Block Data transfer''' </center>]]

=== Implementation ===
This section includes implementation, but again, not the details, just the high level. For example, you can list the steps it takes to communicate over a sensor, or the steps needed to write a page of memory onto SPI Flash. You can include sub-sections for each of your component implementation.

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion,

Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.
*http://cmucam.org/projects/cmucam5/wiki/Pixy_Serial_Protocol
*http://cmucam.org/projects/cmucam5/wiki/Porting_Guide
*https://learn.sparkfun.com/tutorials/pulse-width-modulation
*http://www.socialledge.com/sjsu/index.php?title=Realtime_OS_on_Embedded_Systems

F16: Pattern Detector

2016-12-20T22:00:53Z

146 user5:

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center>]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: Pixy Camera.jpg| frame | center|<center>'''Figure 2. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File: Pixy Camera Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: Servo Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receive and sending bytes of information, and the run function that executes the byte transfer between the Pixy Camera and the SJOne board. Below is the Pseudocode an explanation for each function utilized within the UART driver.

<center><big> ''Initialization''</big> </center>

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below. From this equation we derive the DLL and DLM registers which are used as the divisors to establish the desired baud rate for the data transfer. The DLAB bit must also be initialized at is used as both the Receiver Buffer Register and the Transmit Holding Register. Both of which are set at the LCR register. The LCR register is also responsible for setting the start and stop bits of the UART communication.

[[File: Baud Rate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

''Initialization PseudoCode''

PCONP = Power on UART2
PCLKSEL1 = Set Peripheral Clock
PINSEL4 = Set RX and TX pin functionalities
// Acquire DLL_Value from desired Baud Rate
DLL_Value = clock/(16*Baud)
Set DLM and DLL registers according to DLL_Value
LCR = Set 8 bit data transfer Length
LCR = Set start and stop bit for data transfer

<center><big> ''Receiving Data''</big> </center>

Since the SJOne board is only receiving data from the Pixy Camera and not sending data in this situation only a receive byte function is required. The function waits to see if the data is transmitted from the Pixy Camera to the SJOne board by checking the RDR (Receiver Data Ready) register. Once the value of this register is set to 1, the receive function can then capture the transmitted data by looking at the RBR (Receiver Buffer Register).

''Receiving Function PseudoCode''

while (1){
if(RDR){
break;
}
}
new_data = RBR;

=== Pixy Camera Software Implementation ===

The Pixy Camera is able to externally communicate with micro-controllers by sending data regarding the object it is tracking, as well as receiving data for controlling the Pixy Camera itself. When the Pixy camera locates a recognized or preprogrammed object it begins to send information to a connected peripheral in bytes regarding that object. The first few bytes the Pixy camera sends are sync bytes, in other words bytes that let the external device know that the Pixy Camera has picked up on an object. The rest of the following bytes indicate the signature of the object, where it is on the screen with respect to the camera view (x/y center), as well as the width and the height of the object. Below are the specific bytes the Pixy Camera send to the SJOne board when it picks up on a device.

[[File: Pixy_ObjectBlock.png| frame | center| <center> '''Figure 7. Pixy Camera Object Block Data transfer''' </center>]]

=== Implementation ===
This section includes implementation, but again, not the details, just the high level. For example, you can list the steps it takes to communicate over a sensor, or the steps needed to write a page of memory onto SPI Flash. You can include sub-sections for each of your component implementation.

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion,

Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.

F16: Pattern Detector

2016-12-20T21:59:54Z

146 user5: /* Conclusion */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center>]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: Pixy Camera.jpg| frame | center|<center>'''Figure 2. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File: Pixy Camera Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: Servo Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receive and sending bytes of information, and the run function that executes the byte transfer between the Pixy Camera and the SJOne board. Below is the Pseudocode an explanation for each function utilized within the UART driver.

<center><big> ''Initialization''</big> </center>

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below. From this equation we derive the DLL and DLM registers which are used as the divisors to establish the desired baud rate for the data transfer. The DLAB bit must also be initialized at is used as both the Receiver Buffer Register and the Transmit Holding Register. Both of which are set at the LCR register. The LCR register is also responsible for setting the start and stop bits of the UART communication.

[[File: Baud Rate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

''Initialization PseudoCode''

PCONP = Power on UART2
PCLKSEL1 = Set Peripheral Clock
PINSEL4 = Set RX and TX pin functionalities
// Acquire DLL_Value from desired Baud Rate
DLL_Value = clock/(16*Baud)
Set DLM and DLL registers according to DLL_Value
LCR = Set 8 bit data transfer Length
LCR = Set start and stop bit for data transfer

<center><big> ''Receiving Data''</big> </center>

Since the SJOne board is only receiving data from the Pixy Camera and not sending data in this situation only a receive byte function is required. The function waits to see if the data is transmitted from the Pixy Camera to the SJOne board by checking the RDR (Receiver Data Ready) register. Once the value of this register is set to 1, the receive function can then capture the transmitted data by looking at the RBR (Receiver Buffer Register).

''Receiving Function PseudoCode''

while (1){
if(RDR){
break;
}
}
new_data = RBR;

=== Pixy Camera Software Implementation ===

The Pixy Camera is able to externally communicate with micro-controllers by sending data regarding the object it is tracking, as well as receiving data for controlling the Pixy Camera itself. When the Pixy camera locates a recognized or preprogrammed object it begins to send information to a connected peripheral in bytes regarding that object. The first few bytes the Pixy camera sends are sync bytes, in other words bytes that let the external device know that the Pixy Camera has picked up on an object. The rest of the following bytes indicate the signature of the object, where it is on the screen with respect to the camera view (x/y center), as well as the width and the height of the object. Below are the specific bytes the Pixy Camera send to the SJOne board when it picks up on a device.

=== Implementation ===
This section includes implementation, but again, not the details, just the high level. For example, you can list the steps it takes to communicate over a sensor, or the steps needed to write a page of memory onto SPI Flash. You can include sub-sections for each of your component implementation.

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion, we are both satisfied with our project results, but at the same time we know could've done a lot better and put more time into it. Most of the components were able to function correctly, while we had problems with our parts of the project. We are glad we took this class, because it outlines important concepts that we needed to learn and we were able to learn how FREE RTOS works. We were also able to implement our own FREE RTOS for the purpose of our project. Every group comes into problems, but our group was successfully able to solve them and keep on working on our goal to get the project working. If we were do it again, I would recommend have a leader in group, set deadline, make sure people are getting work accomplished, and work together as a team. This project had not only increased my knowlegde about Embedded Systems, but it has also increased my knowlegde on how project and teams work in real life.

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.

F16: Pattern Detector

2016-12-20T21:57:33Z

146 user5: /* Pixy Camera Software Implementation */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center>]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: Pixy Camera.jpg| frame | center|<center>'''Figure 2. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File: Pixy Camera Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: Servo Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receive and sending bytes of information, and the run function that executes the byte transfer between the Pixy Camera and the SJOne board. Below is the Pseudocode an explanation for each function utilized within the UART driver.

<center><big> ''Initialization''</big> </center>

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below. From this equation we derive the DLL and DLM registers which are used as the divisors to establish the desired baud rate for the data transfer. The DLAB bit must also be initialized at is used as both the Receiver Buffer Register and the Transmit Holding Register. Both of which are set at the LCR register. The LCR register is also responsible for setting the start and stop bits of the UART communication.

[[File: Baud Rate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

''Initialization PseudoCode''

PCONP = Power on UART2
PCLKSEL1 = Set Peripheral Clock
PINSEL4 = Set RX and TX pin functionalities
// Acquire DLL_Value from desired Baud Rate
DLL_Value = clock/(16*Baud)
Set DLM and DLL registers according to DLL_Value
LCR = Set 8 bit data transfer Length
LCR = Set start and stop bit for data transfer

<center><big> ''Receiving Data''</big> </center>

Since the SJOne board is only receiving data from the Pixy Camera and not sending data in this situation only a receive byte function is required. The function waits to see if the data is transmitted from the Pixy Camera to the SJOne board by checking the RDR (Receiver Data Ready) register. Once the value of this register is set to 1, the receive function can then capture the transmitted data by looking at the RBR (Receiver Buffer Register).

''Receiving Function PseudoCode''

while (1){
if(RDR){
break;
}
}
new_data = RBR;

=== Pixy Camera Software Implementation ===

The Pixy Camera is able to externally communicate with micro-controllers by sending data regarding the object it is tracking, as well as receiving data for controlling the Pixy Camera itself. When the Pixy camera locates a recognized or preprogrammed object it begins to send information to a connected peripheral in bytes regarding that object. The first few bytes the Pixy camera sends are sync bytes, in other words bytes that let the external device know that the Pixy Camera has picked up on an object. The rest of the following bytes indicate the signature of the object, where it is on the screen with respect to the camera view (x/y center), as well as the width and the height of the object. Below are the specific bytes the Pixy Camera send to the SJOne board when it picks up on a device.

=== Implementation ===
This section includes implementation, but again, not the details, just the high level. For example, you can list the steps it takes to communicate over a sensor, or the steps needed to write a page of memory onto SPI Flash. You can include sub-sections for each of your component implementation.

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion,

Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.

F16: Pattern Detector

2016-12-20T21:42:44Z

146 user5: /* Software Design SJOne Board */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center>]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: Pixy Camera.jpg| frame | center|<center>'''Figure 2. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File: Pixy Camera Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: Servo Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receive and sending bytes of information, and the run function that executes the byte transfer between the Pixy Camera and the SJOne board. Below is the Pseudocode an explanation for each function utilized within the UART driver.

<center><big> ''Initialization''</big> </center>

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below. From this equation we derive the DLL and DLM registers which are used as the divisors to establish the desired baud rate for the data transfer. The DLAB bit must also be initialized at is used as both the Receiver Buffer Register and the Transmit Holding Register. Both of which are set at the LCR register. The LCR register is also responsible for setting the start and stop bits of the UART communication.

[[File: Baud Rate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

''Initialization PseudoCode''

PCONP = Power on UART2
PCLKSEL1 = Set Peripheral Clock
PINSEL4 = Set RX and TX pin functionalities
// Acquire DLL_Value from desired Baud Rate
DLL_Value = clock/(16*Baud)
Set DLM and DLL registers according to DLL_Value
LCR = Set 8 bit data transfer Length
LCR = Set start and stop bit for data transfer

<center><big> ''Receiving Data''</big> </center>

Since the SJOne board is only receiving data from the Pixy Camera and not sending data in this situation only a receive byte function is required. The function waits to see if the data is transmitted from the Pixy Camera to the SJOne board by checking the RDR (Receiver Data Ready) register. Once the value of this register is set to 1, the receive function can then capture the transmitted data by looking at the RBR (Receiver Buffer Register).

''Receiving Function PseudoCode''

while (1){
if(RDR){
break;
}
}
new_data = RBR;

=== Pixy Camera Software Implementation ===

=== Implementation ===
This section includes implementation, but again, not the details, just the high level. For example, you can list the steps it takes to communicate over a sensor, or the steps needed to write a page of memory onto SPI Flash. You can include sub-sections for each of your component implementation.

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion,

Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.

F16: Pattern Detector

2016-12-20T21:38:14Z

146 user5: /* Software Design SJOne Board */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center>]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: Pixy Camera.jpg| frame | center|<center>'''Figure 2. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File: Pixy Camera Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: Servo Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receive and sending bytes of information, and the run function that executes the byte transfer between the Pixy Camera and the SJOne board. Below is the Pseudocode an explanation for each function utilized within the UART driver.

<center><big> ''Initialization''</big> </center>

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below. From this equation we derive the DLL and DLM registers which are used as the divisors to establish the desired baud rate for the data transfer. The DLAB bit must also be initialized at is used as both the Receiver Buffer Register and the Transmit Holding Register. Both of which are set at the LCR register. The LCR register is also responsible for setting the start and stop bits of the UART communication.

[[File: Baud Rate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

''Initialization PseudoCode''

PCONP = Power on UART2
PCLKSEL1 = Set Peripheral Clock
PINSEL4 = Set RX and TX pin functionalities
// Acquire DLL_Value from desired Baud Rate
DLL_Value = clock/(16*Baud)
Set DLM and DLL registers according to DLL_Value
LCR = Set 8 bit data transfer Length
LCR = Set start and stop bit for data transfer

<center><big> ''Receiving Byte Function''</big> </center>

Since the SJOne board is only receiving data from the Pixy Camera and not sending data in this situation only a receive byte function is required. The function waits to see if the data is transmitted from the Pixy Camera to the SJOne board by checking the RDR (Receiver Data Ready) register. Once the value of this register is set to 1, the receive function can then capture the transmitted data by looking at the RBR (Receiver Buffer Register).

''Receiving Function PseudoCode''

while (1){
if(RDR){
break;
}
}
new_data = RBR;

=== Pixy Camera Software Implementation ===

=== Implementation ===
This section includes implementation, but again, not the details, just the high level. For example, you can list the steps it takes to communicate over a sensor, or the steps needed to write a page of memory onto SPI Flash. You can include sub-sections for each of your component implementation.

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion,

Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.

F16: Pattern Detector

2016-12-20T20:49:38Z

146 user5: /* Pixy Camera */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center>]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: Pixy Camera.jpg| frame | center|<center>'''Figure 2. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File: Pixy Camera Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: Servo Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receive and sending bytes of information, and the run function that executes the byte transfer between the Pixy Camera and the SJOne board. Below is the Pseudocode an explanation for each function utilized within the UART driver.

'''''Initialization'''''

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below.

[[File: Baud Rate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

=== Pixy Camera Software Implementation ===

=== Implementation ===
This section includes implementation, but again, not the details, just the high level. For example, you can list the steps it takes to communicate over a sensor, or the steps needed to write a page of memory onto SPI Flash. You can include sub-sections for each of your component implementation.

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion,

Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.

F16: Pattern Detector

2016-12-20T20:49:16Z

146 user5: /* Hardware Design */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center>]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: Pixy Camera.jpg| frame | center|<center>'''Figure 3. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File: Pixy Camera Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: Servo Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receive and sending bytes of information, and the run function that executes the byte transfer between the Pixy Camera and the SJOne board. Below is the Pseudocode an explanation for each function utilized within the UART driver.

'''''Initialization'''''

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below.

[[File: Baud Rate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

=== Pixy Camera Software Implementation ===

=== Implementation ===
This section includes implementation, but again, not the details, just the high level. For example, you can list the steps it takes to communicate over a sensor, or the steps needed to write a page of memory onto SPI Flash. You can include sub-sections for each of your component implementation.

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion,

Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.

F16: Pattern Detector

2016-12-20T20:48:53Z

146 user5: /* Design & Implementation */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram </center]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: Pixy Camera.jpg| frame | center|<center>'''Figure 3. Pixy Camera'''</center> ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File: Pixy Camera Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: Servo Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receive and sending bytes of information, and the run function that executes the byte transfer between the Pixy Camera and the SJOne board. Below is the Pseudocode an explanation for each function utilized within the UART driver.

'''''Initialization'''''

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below.

[[File: Baud Rate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

=== Pixy Camera Software Implementation ===

=== Implementation ===
This section includes implementation, but again, not the details, just the high level. For example, you can list the steps it takes to communicate over a sensor, or the steps needed to write a page of memory onto SPI Flash. You can include sub-sections for each of your component implementation.

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion,

Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.

F16: Pattern Detector

2016-12-20T20:47:25Z

146 user5: /* Pixy Camera */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: Pixy Camera.jpg| frame | center| <center> '''Figure 3. Pixy Camera''' ]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File: Pixy Camera Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: Servo Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receive and sending bytes of information, and the run function that executes the byte transfer between the Pixy Camera and the SJOne board. Below is the Pseudocode an explanation for each function utilized within the UART driver.

'''''Initialization'''''

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below.

[[File: Baud Rate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

=== Pixy Camera Software Implementation ===

=== Implementation ===
This section includes implementation, but again, not the details, just the high level. For example, you can list the steps it takes to communicate over a sensor, or the steps needed to write a page of memory onto SPI Flash. You can include sub-sections for each of your component implementation.

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion,

Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.

F16: Pattern Detector

2016-12-20T20:46:44Z

146 user5: /* Parts List & Cost */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|$8
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: Pixy Camera.jpg| frame | center| <center> '''Figure 3. Pixy Camera''' </center>]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File: Pixy Camera Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: Servo Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receive and sending bytes of information, and the run function that executes the byte transfer between the Pixy Camera and the SJOne board. Below is the Pseudocode an explanation for each function utilized within the UART driver.

'''''Initialization'''''

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below.

[[File: Baud Rate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

=== Pixy Camera Software Implementation ===

=== Implementation ===
This section includes implementation, but again, not the details, just the high level. For example, you can list the steps it takes to communicate over a sensor, or the steps needed to write a page of memory onto SPI Flash. You can include sub-sections for each of your component implementation.

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion,

Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.

F16: Pattern Detector

2016-12-20T20:46:09Z

146 user5: /* Objectives & Introduction */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

*1. Gather information about Pixy Camera and its functionality.
*2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
*3. Create and buy parts for the stand and servos.
*4. Assemble the parts and connect all the elements together.
*5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
*6. Implement the servos with the Pixy Camera.
*7. Test for a different types of functionality.
*8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: Pixy Camera.jpg| frame | center| <center> '''Figure 3. Pixy Camera''' </center>]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File: Pixy Camera Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: Servo Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receive and sending bytes of information, and the run function that executes the byte transfer between the Pixy Camera and the SJOne board. Below is the Pseudocode an explanation for each function utilized within the UART driver.

'''''Initialization'''''

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below.

[[File: Baud Rate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

=== Pixy Camera Software Implementation ===

=== Implementation ===
This section includes implementation, but again, not the details, just the high level. For example, you can list the steps it takes to communicate over a sensor, or the steps needed to write a page of memory onto SPI Flash. You can include sub-sections for each of your component implementation.

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion,

Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.

F16: Pattern Detector

2016-12-20T20:44:05Z

146 user5: /* Conclusion */

== Abstract ==
Camera with object identifying abilities, to find a unique object as well as track it. Once the unique object is recognized by the Pixy Camera, the Camera will then lock the object's position. This will be done by the Pixy Camera sending a signal to the servos, to tell them where the object is within its frame. The microcontroller will then call a different function and control the servos to lock onto the object and track it as well. Overall, the main idea of this project is to create the drivers for the Pixy Camera to find the object and to communicate with the SJOne Board.

== Objectives & Introduction ==
The objective of this project was to create a stand and interface for the Pixy Camera along with SJOne Board to lock onto and track a object. We found the Pixy Camera online, which is created by Charmed Labs, which they made to recognize the outlines of colorful object with certain signatures. We went ahead and created a stand, along with servos so the camera can interface with the microcontroller to control the camera to follow the object. This project has many purposes for tracking objects.

1. Gather information about Pixy Camera and its functionality.
2. Created pins diagrams to connect the servos, Pixy Camera and SJOne Board.
3. Create and buy parts for the stand and servos.
4. Assemble the parts and connect all the elements together.
5. Implment the SPI Interface between the Pixy Camera and SJOne Board.
6. Implement the servos with the Pixy Camera.
7. Test for a different types of functionality.
8. Make sure it locks and tracks the object with 180 degrees of range with Pan or Tilt.

=== Team Members & Responsibilities ===
* Mohit Bhasin
** Make mount and configure Pixy Camera along with the SJOne Board
** Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera
** Make communication between Pixy Camera and servos on the mount for movement
* Nadim Sarras
**Design and Create Hardware connections between micro-controller and external parts.
**Develop SPI Driver to allow for communication between the micro-controller and Pixy Camera

== Schedule ==
A Table of our scheduled as planned before you started working on the project. Then in another table column, so the readers can see the planned vs. actual goals. The point of our schedule is for the readers to assess how to pace themselves if they are using our project as a reference or are doing something similar.

{| class="wikitable"
|-
! scope="col"| Week#
! scope="col"| Start Date
! scope="col"| End Date
! scope="col"| Task
! scope="col"| Completion
|-
! scope="row"| 1
| 10/21/2016
| 10/28/2016
| Finish Proposal and confirm idea
| Completed
|-
! scope="row"| 2
| 10/29/2016
| 11/04/2016
| Draw schematics of final product and determine the materials we need
| Completed
|-
! scope="row"| 3
| 11/05/2016
| 11/11/2016
| Write Drivers for Pixy camera to learn and recognize objects
| Completed
|-
! scope="row"| 4
| 11/12/2016
| 11/18/2016
| Make a mount for the camera that is able to move around and follow objects
| Completed
|-
! scope="row"| 5
| 11/19/2016
| 11/25/2016
| Program Microcontroller to interface with the Pixy Camera, using SPI communication.
| Completed
|-
! scope="row"| 6
| 11/26/2016
| 11/02/2016
| Finish Free RTOS tasks for the camera and movable mount
| Completed
|-
! scope="row"| 7
| 12/03/2016
| 12/09/2016
| Test and Configure Project Accordingly
| Completed
|-
! scope="row"| 8
| 12/10/2016
| 12/16/2016
| Final Tests
| Completed
|}

== Parts List & Cost ==

{| class="wikitable"
! align="center"|Qty
! align="center"|Description
! align="center"|Manufacturer
! align="center"|Part Number
! align="center"|Cost
! align="center"|Links
|-
|1
|SJ One Board [http://www.socialledge.com/sjsu/images/d/de/2012SJOneBoardSchematic.pdf]
|Preet
|SJ-one
|$80
|http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board
|-
|1
|Pixy Camera
|Amazon
|CMUcam5
|$69
|https://www.amazon.com/Pixy-CMUcam5-Smart-Vision-Sensor/dp/B00IUYUA80
|-
|1
|Jumper Wires
|Amazon
|Jumper Wires
|-
|1
|Pan/Tilt Servo Motor Kit
|Amazon
|2 Axis Robotic Camera Mount
|$39
|https://www.amazon.com/gp/product/B00IVOEN1Y/ref=pd_sim_421_1?ie=UTF8&psc=1&refRID=A7PW5YB91E29JTAEV4TP
|-
|}

== Design & Implementation ==
Within the Design & Implementation section, we have included both the hardware design as well as the software design. We used a Pixy Camera and communicated with through UART protocol. We also show how we assembled and created the mount, so the camera can move. The software section includes the logical flow and a diagram of our algorithm.

=== Hardware Design ===
We have included detailed hardware schematics and the interface of each component.

[[File: Hardware Diagram.png| frame | center| <center> Figure 1. Hardware Diagram]]

==== Pixy Camera ====
The Pixy Camera uses a color-based filtering algorithm to detect objects. Pixy can easily connect to lots of different controllers because it supports several interface options, like UART communication that is being used for the project. The information that is being sent out from the Pixy Camera to the SJOneBoard is the sync word (0xaa55), checksum (sum of all 16-bit words 2-6), signature number, x center of object, y center of object, width of object, height of object. All this information lets us know where the object is in frame, how big it is and the color as well.

[[File: Pixy Camera.jpg| frame | center| <center> '''Figure 3. Pixy Camera''' </center>]]

==== Pixy Camera Mount ====
The Pixy Camera Mount was made so the camera had a place to sit on. The mount is very useful when testing or using this camera, because then no one has to hold the camera while using it. Instead the camera can rest on the mount which can be placed anywhere and allows easy access to the camera. This way the camera also has the ability to move left - right and up - down.

[[File: Pixy Camera Mount.jpg| frame | center| <center> '''Figure 3. Pixy Camera Mount''' </center>]]

==== PWM Servos ====
Pulse With Modulation Servos are useful for this project, since they allow the camera to move left - right and up - down. Pulse width modulation allows us to vary how much time the signal is high in an analog and this lets us control how the servo moves. With a pulse in the middle, the servo motor will rest at 90 degrees. With the low pulse, the servo will be at the 0 degree and with a large pulse, the servo will be at 180 degrees. Once we receive information from the Pixy Camera we can then modify the position of the servos to move along with the object.

[[File: Servo Motor.jpg| frame | center| <center> Figure 4. '''PWM Servo Motor''' </center>]]

==== SJOneBoard ====
The SJOneBoard is used as the brains of the project. It communicates with the Camera and is able to receive data about the object. The SJOneBoard also powers the Camera. It is also used to send signals to camera to use the servo motors on the camera, so they can move and follow the object.

[[File: Devtutorial SjOneBoardOverlay.png| frame | center| <center> '''Figure 5. SJOneBoard''' </center>]]

=== Hardware Interface ===
In this section, we describe how our hardware communicates, such as which driver implementation we used on a high level workings of our project.

To communicate with SJOneBoard and the Pixy Camera we are using a UART communication driver. This allows us to transmit bits to the Pixy Camera so we can receive useful bits back from the camera. Once we are able to figure out where the object is within the frame and where it is moving, we can trasmit useful bits back to the Pixy Camera. Since there are servos attached onto the Pixy Camera and its mount, we can use these to move the camera around to follow an object. If the object goes out of frame the Pixy can move around until it finds it or turn an LED on saying that it cannot find the object.

=== Software Design SJOne Board ===
The SJOne board utilizes a UART task in order to communicate with the Pixy Camera. The driver task consists of initializing the registers of the SJOne Board, receive and sending bytes of information, and the run function that executes the byte transfer between the Pixy Camera and the SJOne board. Below is the Pseudocode an explanation for each function utilized within the UART driver.

'''''Initialization'''''

The corresponding UART port must be initialized by powering on the PCONP register. The peripheral clock must then be set to the corresponding value of the peripheral device.
In order for the UART transfer to take place the SJOne board utilizes the Tx and Rx pins which are also set within the initialization block by using the PINSEL Register to program the functionality.
The Baud Rate must match the Pixy Camera Baud Rate of 19200bps. This was set utilizing the equation below.

[[File: Baud Rate.png| frame | center| <center> '''Figure 6. Baud Rate Equation''' </center>]]

=== Pixy Camera Software Implementation ===

=== Implementation ===
This section includes implementation, but again, not the details, just the high level. For example, you can list the steps it takes to communicate over a sensor, or the steps needed to write a page of memory onto SPI Flash. You can include sub-sections for each of your component implementation.

== Testing & Technical Challenges ==
We had a couple of technical challenges while we testing our project. In this section we shall describe the challenges of our project. We will also advise the readers if our project can be started from scratch again or they are trying to achieve similar results.

=== Testing ===
We tested our project in different in parts. First we had tested the UART communication of the SJOneBoard with another SJOneBoard to see if the UART protocol was working properly. Once we were successfully able to transmit and receive bits, we moved onto the next step. Although the driver for UART is already written we have to make changes for the purpose of our project. We made another RTOS Task for the new UART for communication with the Pixy and the Pixy's servos. Once we completed this we tested to see if we could properly read the bits from Pixy Camera through UART interface. We used a Saleae to see the bits that were being transferred. Then we could control the servos on the Pixy once we received the correct information. We used test cases with one object and then we moved to two objects for the purpose of this project. We noticed it could see all the objects, but could only track one, which would be the most sensitive object.

=== My Issue #1 ===
Establishing communication between the Pixy Camera and the SJOne micro-controller proved to be the most difficult task. The Pixy Camera micro-controller was configured and manufactured for Arduino capabilities. However no one has ever established communication between the Pixy Camera and the SJOne board before. The team initially attempted to establish a SPI interface between the Pixy Camera and the SJOne board. However, after multiple failed attempts, most likely caused by the different clock rates, the team then switched over to UART communication.

=== My Issue #2 ===
Another issue we ran into was to gain access to utilize the Pixy's servo controls. Since we have attached two servos on the back of the camera and mounted to the mount. We want to control them so we can move the Pixy camera to follow the object. Apparently you have to send certain bytes to tell the Pixy you want to gain access to its servo controls. Then we have to send the bytes to tell the Pixy how to pulse the servos, so they are in the positions we want them to be in.

== Conclusion ==
In conclusion,

Conclude your project here. You can recap your testing and problems. You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?

=== Project Video ===
We have uploaded the video onto YouTube and the link below will allow you access to the video.

=== Project Source Code ===
* [https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]

== References ==
=== Acknowledgement ===
Professors
* Preet
* Dr Ozemek

Lab Assistants
* Praveen Prabhakaran
* Shilpa Srinivasan

=== References Used ===
List any references used in project.

F16: Pattern Detector

2016-12-20T20:37:39Z

146 user5: /* Testing & Technical Challenges */

F16: Pattern Detector

2016-12-20T20:23:54Z

146 user5:

F16: Pattern Detector

2016-12-20T20:20:42Z

146 user5:

F16: Pattern Detector

2016-12-20T20:10:20Z

146 user5: /* Testing & Technical Challenges */