Matteo Spanio

Analog audio documents inevitably face degradation over time, posing a challenge for preserving their audio content and ensuring the integrity of the recordings. Analog document preservation is one of the main research topics of interest of the Centro di Sonologia Computazionale (CSC) of the Department of Information Engineering of the University of Padua, which over the years developed and implemented a methodology for preservation that includes, among other things, the video recording of the digitization process of the open-reel tapes for documenting irregularities on the top of their surface. Together with the corpus of digitized high-quality audio recordings, this led to the creation of an internal archive of video documents. This paper presents a software application that leverages computer vision techniques to automatically detect Irregularities on open-reel audio tapes, analyzing the video documents produced during the digitization interventions. The software employs a frame-by-frame analysis to automatically identify and highlight points of interest that may indicate tape damages, splices, and other Irregularities. The software uses Generalized Hough Transform and SURF algorithms to locate regions of interest within the tape. The proposed software is also part of the MPAI/IEEE-CAE ARP standard developed by Audio Innova s.r.l., spin-off of the CSC, and it may offer a robust and efficient solution for analyzing open-reel audio tapes, supporting archivists and musicologists in their activities.

The Moving Picture, Audio and Data Coding by Artificial Intelligence (MPAI) Context-based Audio Enhancement (CAE) Audio Recording Preservation (ARP) standard provides the technical specifications for a comprehensive framework for digitizing and preserving analog audio, specifically focusing on documents recorded on open-reel tapes. This paper presents a novel envelope derivative-based method designed to be integrated into the ARP standard, for detecting reverse audio sections during the preservation process. The primary objective of this method is to automatically identify segments of audio recorded in reverse. Leveraging derivative-based signal processing algorithms, the system enhances its capability to detect and reverse such sections, thereby reducing errors during the preservation process. This feature not only aids in identifying and correcting errors but also enhances the efficiency of large-scale audio document archiving projects. The system’s performance was evaluated using a diverse dataset that includes various musical genres and digitized tapes, demonstrating its strong potential and effectiveness across different types of audio content.

In recent decades, neuroscientific and psychological research has traced direct relationships between taste and auditory perceptions. This article explores multimodal generative models capable of converting taste information into music, building on this foundational research. We provide a brief review of the state of the art in this field, highlighting key findings and methodologies. We present an experiment in which a fine-tuned version of a generative music model (MusicGEN) is used to generate music based on detailed taste descriptions provided for each musical piece. The results are promising: according the participants’ (n=111) evaluation, the fine-tuned model produces music that more coherently reflects the input taste descriptions compared to the non-fine-tuned model. This study represents a significant step towards understanding and developing embodied interactions between AI, sound, and taste, opening new possibilities in the field of generative AI. We release our dataset, code and pre-trained model at: https://osf.io/xs5jy/.