Piano + Sheet Music Teacher
This project is intended to teach people the basics of reading music as instructions for playing piano. It involves displaying some sheet music on LCD screens and modifying a piano keyboard to include LEDs on keys. The note that the user should play is indicated on the sheet music by its color change to green, and its associated LED on the keyboard is lit. Once the user presses the correct key, the music and LEDs change to indicate the next note. This was created through the collaboration of Jai Chauhan, Jillian Kovalcson, and Frances Tsenn for their ECE 4180 class.
Parts list
- Mbed
- Piano keyboard
- 2 uLCD screens
- 25 green LEDs
- 25 resistors (47 Ω)
- 2 SPI MCP23S17 chips
About the Keyboard
A large portion of the project involved reading and understanding signals from the capacitive touch sensors underneath each piano key, and there is likely a large amount of variability for this depending on the keyboard chosen. In the keyboard used for the project, there was a first set of 5 wires and a second set of 8 wires. The set of 5 wires indicates which region of the keyboard includes the pressed key. The set of 8 wires indicates which key in the region was pressed, which is displayed as different colored keys in the above figure. We refer to the set of 5 as the "rows" and the set of 8 as the "columns" when indicating which combination was needed for a key. By default, the values of both sets are high. When a key is pressed, one row and one column signal is pulled low.
Connecting the Components
SPI-L (Left on Keyboard)
Mbed | MCP23S17 |
---|---|
GND | A0, A1, A2, Vss |
3.3V | /CS, /RESET, Vdd |
P5 | SI |
P6 | SO |
P7 | SCK |
SPI-R (Right on Keyboard)
Mbed | MCP23S17 |
---|---|
GND | A1, A2, Vss |
3.3V | A0, /CS, /RESET, Vdd |
P11 | SI |
P12 | SO |
P13 | SCK |
uLCD-L (Left on Keyboard)
Mbed | uLCD header | uLCD cable |
---|---|---|
5V=VU | 5V | 5V |
GND | GND | GND |
TX=P9 | RX | TX |
RX=P10 | TX | RX |
P11 | Reset | Reset |
uLCD-R (Right on Keyboard)
Mbed | uLCD header | uLCD cable |
---|---|---|
5V=VU | 5V | 5V |
GND | GND | GND |
TX=P28 | RX | TX |
RX=P27 | TX | RX |
P29 | Reset | Reset |
To fit within the keyboard, the LEDs and resistors were modified. On the negative lead of the LED, a resistor was soldered so that the end of the resistor was within 2cm of the LED. The resistor/LED lead was covered with approximately a 2cm long piece of shrink tubing. The positive lead of the LED was soldered to a solid wire.
The LED was placed within the drilled holes in the keys of the piano. The resistors were connected to a ground wire that went along the length of the keyboard.
Source code
Import programECE4180PianoTutorial
Piano tutorial
The song used is defined in music.h. The hex value used are the row number and column number associated with each key. Rows are numbered 1 through 5, with 1 being at the left side of the keyboard and 5 being at the right. Each row is associated with columns numbered 1 through 8 arranged in order of increasing pitch. The duration is a 1 if the note is a quarter note and 2 if the note is a half note.
music.h
#ifndef MUSIC_H #define MUSIC_H //twinkle twinkle little star, one verse. //CCGG|AAG-|FFEE|DDC-|GGFF|EED-|GGFF|EED-|CCGG|AAG-|FFEE|DDC-| const int notes[42] = {0x1DEF, 0x1DEF, 0x1BF7, 0x1BF7, 0x1BDF, 0x1BDF, 0x1BF7, 0x1BFD, 0x1BFD, 0x1BFE, 0x1BFE, 0x1DBF, 0x1DBF, 0x1DEF, 0x1BF7, 0x1BF7, 0x1BFD, 0x1BFD, 0x1BFE, 0x1BFE, 0x1DBF, 0x1BF7, 0x1BF7, 0x1BFD, 0x1BFD, 0x1BFE, 0x1BFE, 0x1DBF, 0x1DEF, 0x1DEF, 0x1BF7, 0x1BF7, 0x1BDF, 0x1BDF, 0x1BF7, 0x1BFD, 0x1BFD, 0x1BFE, 0x1BFE, 0x1DBF, 0x1DBF, 0x1DEF }; const int lengths[42] = { 1, 1, 1, 1, 1, 1, 2, 1, 1, 1, 1, 1, 1, 2, 1, 1, 1, 1, 1, 1, 2, 1, 1, 1, 1, 1, 1, 2, 1, 1, 1, 1, 1, 1, 2, 1, 1, 1, 1, 1, 1, 2 }; const int endBar[42] = {0,0,0,1,0,0,1, 0,0,0,1,0,0,1, 0,0,0,1,0,0,1, 0,0,0,1,0,0,1, 0,0,0,1,0,0,1, 0,0,0,1,0,0,1 }; #endif
A majority of the functional code needed to run on the mbed is within the main.cpp file. This includes everything needed to receive input from the keyboard, writing to the LEDs, and modifying the 2 uLCD displays.
main.cpp
#include "mbed.h" #include "uLCD_4DGL.h" #include "MCP23S17.h" #include "Music.h" //pinout declarations // Console output Serial pc(USBTX,USBRX); BusIn rKeysBus(p16, p17, p18); //2, 3, 4 BusIn cKeysBus(p19, p20, p21, p22, p23, p24, p25, p26); //1, 2, 3, 4, 5, 6, 7, 8 uLCD_4DGL uLCD1(p9,p10,p8); uLCD_4DGL uLCD2(p28,p27,p29); SPI spiLeft(p5, p6, p7); //SI, SO, SCK SPI spiRight(p11, p12, p13); char OpcodeLeft = 0x40; //A0, A1, A2 all hooked to ground char OpcodeRight = 0x42; //A0 pulled high (3v), A1, A2 are grounded // mbed p14 is connected to ~chipSelect on the MCP23S17 MCP23S17 chipLeft = MCP23S17(spiLeft, p14, OpcodeLeft); // mbed p15 is connected to ~chipSelect on the MCP23S17 MCP23S17 chipRight = MCP23S17(spiRight, p15, OpcodeRight); DigitalOut audioCue(p30);//audio workaround since the piano speaker broke //debug Leds DigitalOut led1(LED1); DigitalOut led2(LED2); DigitalOut led3(LED3); DigitalOut led4(LED4); //Global Variables //LCD variables int LCDMusicColumns[4] = {30,53,76,99}; int LCDMusicLines[8] = {15,30,45,60,75,90,105,120}; int LCDMusicSpaces[7] = {22,37,52,67,82,97,112}; int barLength; int barCount; //30,53,76,99 columns //line rows A=15,F=30,D=45,B=60,G=75,E=90,C=105,A=120 //between rows G=22,E=37,C=52,A=67,F=82,D=97,B=112 //need extra lines - A=15, C=105, B=112, A =120 int musicNoteRows[25] = {120,0,112,105,0,97,0,90,82,0,75,0,67,0,60,52,0,45,0,37,30,0,22,0,15}; //Function Declarations int read_rows(); int read_cols(); int draw_green_note(int note); int draw_bar(int LCDnum); void LCD_startup(); int draw_note(int inputValue, int displayColumn, int flatOrSharp, int noteType, int LCDnum, int color); int find_key_display_row(int inputValue, int flatOrSharp); int find_key_index(int inputValue); int LCD_init(int LCDnum); int draw_extra_staff_line(int x, int y, int color, int LCDnum); int draw_quarter_note(int x, int y, int color,int LCDnum); int draw_half_note(int x, int y, int color, int LCDnum); int draw_main_lines(int LCDnum); //int rNeeded = 0x0F; //options: 0001 1110 (0x1E), 0001 1101 (0x1D), 0001 1011 (0x1B), 0001 0111 (0x17), 0000 1111 (0x0F) //int cNeeded = 0xFE; //options: 1111 1110 (0xFE), 1111 1101 (0xFD), 1111 1011 (0xFB), 1111 0111 (0xF7), // 1110 1111 (0xEF), 1101 1111 (0xDF), 1011 1111 (0xBF), 0111 1111 (0x7F) int main() { rKeysBus.mode(PullUp); cKeysBus.mode(PullUp); LCD_startup();//blank screens int count = 0; int currentNote = notes[count]; int rNeeded = currentNote >> 8; int cNeeded = currentNote & 0xFF; //only 1 & for this // Set all Port bits to output direction chipLeft.direction(PORT_A, 0x00); chipLeft.direction(PORT_B, 0x00); chipRight.direction(PORT_A, 0x00); chipRight.direction(PORT_B, 0x00); //setup the LCDs int valid; barCount = 0; valid = draw_bar(1); if(!valid) { pc.printf("error drawing notes to LCD1\n\r"); } valid = draw_bar(2); if(!valid) { pc.printf("error drawing notes to LCD2\n\r"); } valid = draw_green_note(count); if(!valid) { pc.printf("error drawing green note\n\r"); } //turn on correct LED switch (rNeeded) { case 0x1D: chipLeft.write(PORT_A, ~cNeeded); chipLeft.write(PORT_B, 0x00); chipRight.write(PORT_A, 0x00); chipRight.write(PORT_B, 0x00); break; case 0x1B: chipLeft.write(PORT_A, 0x00); chipLeft.write(PORT_B, ~cNeeded); chipRight.write(PORT_A, 0x00); chipRight.write(PORT_B, 0x00); break; case 0x17: chipLeft.write(PORT_A, 0x00); chipLeft.write(PORT_B, 0x00); chipRight.write(PORT_A, ~cNeeded); chipRight.write(PORT_B, 0x00); break; case 0x0F: chipLeft.write(PORT_A, 0x00); chipLeft.write(PORT_B, 0x00); chipRight.write(PORT_A, 0x00); chipRight.write(PORT_B, ~cNeeded); break; } int rKeys; int cKeys; while(1) { //check if it's the correct key rKeys = (rKeysBus.read() << 1) | 0x11; cKeys = cKeysBus.read(); // only checking columns for now, need to figure out the rows. if (rKeys == rNeeded & cKeys == cNeeded) { audioCue = 1; led2 = 1; //get next note count++; count = count % 42; currentNote = notes[count]; rNeeded = currentNote >> 8; cNeeded = currentNote & 0xFF; //modify uLCD //if at end of bar, redraw screens if(endBar[count-1]) { barCount++; uLCD1.cls(); uLCD2.cls(); draw_main_lines(1); draw_main_lines(2); valid = draw_bar(1); if(!valid) { pc.printf("error drawing notes to LCD1\n\r"); } valid = draw_bar(2); if(!valid) { pc.printf("error drawing notes to LCD2\n\r"); } } //draw the green note valid = draw_green_note(count); if(!valid) { pc.printf("error drawing green note\n\r"); } //turn on correct LED switch (rNeeded) { case 0x1D: chipLeft.write(PORT_A, ~cNeeded); chipLeft.write(PORT_B, 0x00); chipRight.write(PORT_A, 0x00); chipRight.write(PORT_B, 0x00); break; case 0x1B: chipLeft.write(PORT_A, 0x00); chipLeft.write(PORT_B, ~cNeeded); chipRight.write(PORT_A, 0x00); chipRight.write(PORT_B, 0x00); break; case 0x17: chipLeft.write(PORT_A, 0x00); chipLeft.write(PORT_B, 0x00); chipRight.write(PORT_A, ~cNeeded); chipRight.write(PORT_B, 0x00); break; case 0x0F: chipLeft.write(PORT_A, 0x00); chipLeft.write(PORT_B, 0x00); chipRight.write(PORT_A, 0x00); chipRight.write(PORT_B, ~cNeeded); break; } if (notes[count] == notes[count - 1]) { //wait on repeat notes wait(0.2); } audioCue = 0; } else { //ends if for correct key led2 = 0; }; } } // helper function ----------------------------------------------------------------- int draw_green_note(int note) { //pc.printf("Drawing Green Note\n\r"); int valid; int count; if(barCount%2 == 0) { if(barCount > 1) { count = (barCount/2) * 7; } else { count = 0; } } else { count = ((barCount+1)/2) * 7 - 3; } if (!endBar[note-1]) { if(lengths[note-1] == 1) { valid = draw_note(notes[note-1], note - count, 0, lengths[note-1], 1, BLACK); } else if (lengths[note-1] == 2) { valid = draw_note(notes[note-1], note - count - 1, 0, lengths[note-1], 1, BLACK); } if(!valid) { return 0; } } valid = draw_note(notes[note], note - count + 1, 0, lengths[note], 1, GREEN); if (!valid) { return 0; } return 1; } int draw_bar(int LCDnum) { //pc.printf("Drawing the bar on LCD# %d \n\r",LCDnum); int valid; barLength = 0; if(LCDnum == 2) { barCount++; } //pc.printf("BarCount: %d",barCount); if(barCount < 12) { int count; if(barCount%2 == 0) { if(barCount > 1) { count = (barCount/2) * 7; } else { count = 0; } } else { count = ((barCount+1)/2) * 7 - 3; } //pc.printf("Count: %d \n\rLCD: %d \n\r",count, LCDnum); if(count !=0) { while(!endBar[count]) { valid = draw_note(notes[count], barLength + 1, 0, lengths[count], LCDnum, BLACK); if(!valid) { pc.printf("error drawing note\n\r"); return 0; } barLength += lengths[count]; count++; } valid = draw_note(notes[count], barLength + 1, 0, lengths[count], LCDnum, BLACK); if(!valid) { pc.printf("error drawing note\n\r"); return 0; } } else { for (int i = 0; i<4; i++) { valid = draw_note(notes[count], barLength + 1, 0, lengths[count], LCDnum, BLACK); if(!valid) { pc.printf("error drawing note\n\r"); return 0; } barLength += lengths[count]; count++; } } } else { LCD_init(LCDnum); } if(LCDnum == 2) { barCount--; } return 1; } void LCD_startup() { LCD_init(1); LCD_init(2); draw_main_lines(1); draw_main_lines(2); } int draw_note(int inputValue, int displayColumn, int flatOrSharp, int noteType, int LCDnum, int color) { //inputValue is the hexcode representing the note //display column = 1,2,3 or 4 this is the note's position in the bar //flatorsharp = 0 is natural,1 is flat, 2 is sharp. //notetype = 1 is quarter note, 2 is halfnote //pc.printf("Drawing this note -- Column: %d NoteType: %d Color: %d\n\r",displayColumn,noteType,color); int thisNoteRow = find_key_display_row(inputValue, flatOrSharp); int valid; if(noteType == 1) { valid = draw_quarter_note(LCDMusicColumns[displayColumn - 1],thisNoteRow,color,LCDnum); if(!valid) { pc.printf("error drawing quarter note\n\r"); } } else if (noteType == 2) { valid = draw_half_note(LCDMusicColumns[displayColumn - 1],thisNoteRow,color,LCDnum); if(!valid) { pc.printf("error drawing half note\n\r"); } } //draw the extra lines for A above the staff and C, B, and A below the staff switch(thisNoteRow) { case 15: valid = draw_extra_staff_line(LCDMusicColumns[displayColumn-1],thisNoteRow, color, LCDnum); if(!valid) { pc.printf("error drawing line above the staff \n\r"); } break; case 120: valid = draw_extra_staff_line(LCDMusicColumns[displayColumn-1], thisNoteRow, color, LCDnum); if(!valid) { pc.printf("error drawing line below the staff \n\r"); } case 112: case 105: valid = draw_extra_staff_line(LCDMusicColumns[displayColumn-1], 105, color, LCDnum); if(!valid) { pc.printf("error drawing line below the staff \n\r"); } break; default: break; } if(flatOrSharp > 0) { if(flatOrSharp == 1) { //flat //flat figure [10x6] x-10, y-2 int x = LCDMusicColumns[displayColumn - 1] - 10; int y = thisNoteRow - 2; if(LCDnum == 1) { uLCD1.line(x-3,y-5,x-3,y+6,BLACK); uLCD1.line(x-2,y+5,x+3,y,BLACK); uLCD1.line(x-2,y+1,x+1,y-2,BLACK); uLCD1.line(x+1,y-1,x+2,y+1,BLACK); } else if (LCDnum == 2) { uLCD2.line(x-3,y-5,x-3,y+6,BLACK); uLCD2.line(x-2,y+5,x+3,y,BLACK); uLCD2.line(x-2,y+1,x+1,y-2,BLACK); uLCD2.line(x+1,y-1,x+2,y+1,BLACK); } else { pc.printf("Invalid LCD number \n\r"); } } if(flatOrSharp == 2) { //sharp //sharp figure [10x10] x-10 int x = LCDMusicColumns[displayColumn - 1] - 10; int y = thisNoteRow; if(LCDnum == 1) { uLCD1.line(x-3,y-5,x-1,y+5,BLACK); uLCD1.line(x,y-5,x+2,y+5,BLACK); uLCD1.line(x-5,y-1,x+5,y-4,BLACK); uLCD1.line(x-5,y+3,x+5,y,BLACK); } else if (LCDnum == 2) { uLCD2.line(x-3,y-5,x-1,y+5,BLACK); uLCD2.line(x,y-5,x+2,y+5,BLACK); uLCD2.line(x-5,y-1,x+5,y-4,BLACK); uLCD2.line(x-5,y+3,x+5,y,BLACK); } else { pc.printf("Invalid LCD number \n\r"); } } } return 1; } int find_key_display_row(int inputValue, int flatOrSharp) { //natural = 0, flat = 1, sharp = 2 int index = find_key_index(inputValue); //pc.printf("Index %d\n\r", index); if(index < 0) { pc.printf("LCD Display error, incompatible input data \n\r"); return -1; } else { //pc.printf("MusicNoteRow: %d\n\r", musicNoteRows[index]); if(musicNoteRows[index] == 0) { if (flatOrSharp == 1) { return musicNoteRows[index + 1]; } else if (flatOrSharp == 2) { return musicNoteRows[index - 1]; } } else { return musicNoteRows[index]; } } return -1; } int find_key_index(int inputValue) { bool go = true; bool go2 = true; int col = 1; int row = 1; int index = -1; int workingValue = inputValue & 0xFF; while(go) { if(workingValue & 0x0001 == 0x0001) { col++; workingValue = workingValue >> 1; } else { go = false; } } workingValue = inputValue >> 8; while(go2) { if(workingValue & 0x0001 == 0x0001) { row++; workingValue = workingValue >> 1; } else { go2 = false; } } if(col < 9 && row < 6) { index = 8*(row-1) + col - 10; } if(index >= 0 && index < 25) { return index; } else { return -1; } } int LCD_init(int LCDnum) { if(LCDnum == 1) { uLCD1.baudrate(300000); uLCD1.color(BLACK); uLCD1.background_color(WHITE); uLCD1.cls(); return 1; } else if(LCDnum == 2) { uLCD2.baudrate(300000); uLCD2.color(BLACK); uLCD2.background_color(WHITE); uLCD2.cls(); return 1; } else { pc.printf("Invalid LCD number \n\r"); return 0; } } int draw_extra_staff_line(int x, int y, int color, int LCDnum) { if(LCDnum == 1) { uLCD1.line(x-8,y,x+9,y,color); return 1; } else if(LCDnum == 2) { uLCD2.line(x-8,y,x+9,y,color); return 1; } else { pc.printf("Invalid LCD number \n\r"); return 0; } } int draw_quarter_note(int x, int y, int color,int LCDnum) { //pc.printf("LCDnum: %d Quarter note at x: %d y: %d \n\r", LCDnum,x, y); if(LCDnum == 1) { //printf("Note %d %d; \n\r", x, y); if(x>5) { if (y > 30) { uLCD1.filled_circle(x,y,5,color); uLCD1.line(x+5,y,x+5,y-30,color); return 1; } else if (y <=30 && y>0) { uLCD1.filled_circle(x,y,5,color); uLCD1.line(x+5,y,x+5,y+30,color); return 1; } else { return 0; } } else { return 0; } } else if (LCDnum == 2) { //pc.printf("Note %d %d; \n\r", x, y); if(x>5) { if (y > 30) { uLCD2.filled_circle(x,y,5,color); uLCD2.line(x+5,y,x+5,y-30,color); return 1; } else if (y <=30 && y>0) { uLCD2.filled_circle(x,y,5,color); uLCD2.line(x+5,y,x+5,y+30,color); return 1; } else { return 0; } } else { return 0; } } else { //pc.printf("Invalid LCD number \n\r"); return 0; } } int draw_half_note(int x, int y, int color, int LCDnum) { //pc.printf("LCDnum: %d Half note at x: %d y: %d \n\r", LCDnum,x, y); if(LCDnum == 1) { if(x>5) { if(y>30) { if (color == WHITE) { uLCD1.filled_circle(x,y,5,color); uLCD1.line(x+5,y,x+5,y-30,color); return 1; } else { uLCD1.filled_circle(x,y,5,color); uLCD1.filled_circle(x,y,3,WHITE); uLCD1.line(x+5,y,x+5,y-30,color); return 1; } } else if (y<=30) { if (color == WHITE) { uLCD1.filled_circle(x,y,5,color); uLCD1.line(x+5,y,x+5,y+30,color); return 1; } else { uLCD1.filled_circle(x,y,5,color); uLCD1.filled_circle(x,y,3,WHITE); uLCD1.line(x+5,y,x+5,y+30,color); return 1; } } else { return 0; } } else { return 0; } } else if (LCDnum == 2) { if(x>5) { if(y>30) { if (color == WHITE) { uLCD2.filled_circle(x,y,5,color); uLCD2.line(x+5,y,x+5,y-30,color); return 1; } else { uLCD2.filled_circle(x,y,5,color); uLCD2.filled_circle(x,y,3,WHITE); uLCD2.line(x+5,y,x+5,y-30,color); return 1; } } else if (y<=30) { if (color == WHITE) { uLCD2.filled_circle(x,y,5,color); uLCD2.line(x+5,y,x+5,y+30,color); return 1; } else { uLCD2.filled_circle(x,y,5,color); uLCD2.filled_circle(x,y,3,WHITE); uLCD2.line(x+5,y,x+5,y+30,color); return 1; } } else { return 0; } } else { return 0; } } else { pc.printf("Invalid LCD number \n\r"); return 0; } } int draw_main_lines(int LCDnum) { if(LCDnum == 1) { for (int i=0; i<5; i++) { uLCD1.line(0,LCDMusicLines[i+1],128,LCDMusicLines[i+1],BLACK); uLCD1.line(0,LCDMusicLines[i+1]- 1,128,LCDMusicLines[i+1]- 1,BLACK); uLCD1.line(0,29,128,29,BLACK); } return 1; } else if (LCDnum == 2) { for (int i=0; i<5; i++) { uLCD2.line(0,LCDMusicLines[i+1],128,LCDMusicLines[i+1],BLACK); uLCD2.line(0,LCDMusicLines[i+1]- 1,128,LCDMusicLines[i+1]- 1,BLACK); uLCD2.line(0,29,128,29,BLACK); } return 1; } else { pc.printf("Invalid LCD number \n\r"); return 0; } }
Photos & Video Demo
Please log in to post comments.