Research with Prof. Yung-Hsiang Lu · Aug–Dec 2025
Piano transcription
Computer-vision transcription of piano video to MIDI, used to evaluate AI-generated performance videos.
- PyTorch
- OpenCV
- MIDI
Problem
The lab was testing whether a model could generate a realistic video of a piano performance from sheet music. To score those videos, the generated performance had to be turned back into notes and compared with the score. I worked on that transcription step.
What I built
- Reproduced and adapted a visual piano-transcription pipeline (PPAN), first locally and then on Purdue’s Anvil GPU cluster.
- Ran it on the Rach3 and PianoYT datasets and on my own keyboard footage, and connected its output to the lab’s evaluation metrics: note accuracy, onset error, and MIDI similarity.
Results
- datasets
- Rach3, PianoYT
- Purdue research conference
- Fall 2025
Limits
Transcription worked, but accuracy was low. The results showed how hard it is for video-generation models to stay consistent over a long performance, and the lab moved toward other generation methods.