Web sonification with synesthesia tools

Roberto Piassi Passos Bodo; Flávio Luiz Schiavoni

doi:10.5216/mh.v18i1.53572

Authors

Roberto Piassi Passos Bodo University of São Paulo, São Paulo, São Paulo, Brazil http://orcid.org/0000-0002-9316-7672 (unauthenticated)
Flávio Luiz Schiavoni Federal University of São João del-Rei,São João del-Rei, Minas Gerais, Brazil http://orcid.org/0000-0002-0703-3089 (unauthenticated)

DOI:

https://doi.org/10.5216/mh.v18i1.53572

Keywords:

Sonification. HTML5, Web audio, Sonificação estética

Abstract

In this paper, we present an investigation on sonification of web pages involving three media: HTML struc- ture, page text and images. In our approach, HTML pages are read as musical scores; text elements are also read as musical scores following the idea present in notation systems like ABC; and images are read as wave files. We deve- loped a set of tools for website sonification which can be used to explore musical creativity and to create new soun- ds based on the web pages content. In addition, we show how to create sounds and visual feedback using this tool.

Downloads

Download data is not yet available.

Author Biographies

Roberto Piassi Passos Bodo, University of São Paulo, São Paulo, São Paulo, Brazil

Roberto Piassi Passos Bodo (University of São Paulo, São Paulo, São Paulo, Brazil) rppbodo@ime.usp.br
Flávio Luiz Schiavoni, Federal University of São João del-Rei,São João del-Rei, Minas Gerais, Brazil

Flávio Luiz Schiavoni (Federal University of São João del-Rei,São João del-Rei, Minas Gerais, Brazil) fls@ufsj.edu.br

References

ALLISON, Jesse; HOLMES, Daniel; BERKOWITZ, Zachary; PFALZ, Andrew; CONLIN, William; HWANG, Nick; TAYLOR, Benjamin. Programming Music Camp: Using Web Audio to Teach Creative Coding. In: WEB AUDIO CONFERENCE, 2, 2016, Atlanta, Georgia. Proceedings of the 2nd Web Audio Conference. Georgia Institute of Technology, 2016.

BEN-TAL, Oded; BERGER, Jonathan. Creative aspects of sonification. Leonardo, Volume 37, Is- sue 3, p. 229-233, June 2004.

BROWN, Lorna; BREWSTER, Stephen. Drawing by ear: Interpreting sonified line graphs. In: INTERNATIONAL CONFERENCE ON AUDITORY DISPLAY, 9, 2003, Boston, Massachusetts. Proceedings of the 9th International Conference on Auditory Display. Georgia Institute of Tech- nology, 2003.

CARLTON, Dave. I analyzed the chords of 1300 popular songs for patterns. This is what I found. 2012. Available on: <http://www.hooktheory.com/blog/i-analyzed-the-chords-of-1300-popular- -songs-for-patterns-this-is-what-i-found/>. Accessed: 2017-07-01.

DANNENBERG, Roger. An on-line algorithm for real-time accompaniment. In: INTERNATIO- NAL COMPUTER MUSIC CONFERENCE, 1984, France. Proceedings of the 1984 International Computer Music Conference. Computer Music Association, 1985, p. 193-198.

FLOWERS, John; HAUER, Terry. “Sound” alternatives to visual graphics for exploratory data analysis. Behavior Research Methods, Instruments & Computers, Volume 25, Issue 2, p. 242-

??89

??

, June 1993.

GOOD, Michael. MusicXML for Notation and Analysis. In: HEWLETT, Walter; SELFRIDGE- -FIELD, Eleanor. The Virtual Score: Representation, Retrieval, Restoration, MIT Press, Cambrid- ge, MA, 2001, p. 113-124.

HERMANN, Thomas; HUNT, Andy. An Introduction to Interactive Sonification. IEEE MultiMe- dia, Volume 12, Issue 2, p.20-24, April-June 2005.

HERMANN, Thomas; HUNT, Andy; NEUHOFF, John. The Sonification Handbook. Berlin: Lo- gos Publishing House, 2011.

JANA, Reena; TRIBE, Mark. New Media Art. London: Taschen, 2006. 96p.

LAZZARINI, Victor; YI, Steven; TIMONEY, Joseph; KELLER, Damian; PIMENTA, Marcelo. The Mobile CSound Platform. In: INTERNATIONAL COMPUTER MUSIC CONFERENCE, 2012, Slovenia. Proceedings of the 2012 International Computer Music Conference. Computer Music Association, 2012, p. 163-167.

LETZ, Stephane; DENOUX, Sarah; ORLAREY, Yann; FOBER, Dominique. Faust audio DSP lan- guage in the Web. In: Proceedings of the Linux Audio Conference. Mainz, Germany: Johannes Gutenberg University (JGU), 2015, p. 29–36.

MCCRINDLE, Rachel; SYMONS, David. Audio space invaders. In: INTERNATIONAL CONFE- RENCE ON DISABILITY, VIRTUAL REALITY AND ASSOCIATED TECHNOLOGIES, 3, 2000, Alghero, Italy. Proceedings of the 3rd International Conference on Disability, Virtual Reality and Associated Technologies. 2000, p.59-65.

NIENHUYS, Han-Wen; NIEUWENHUIZEN, Jan. Lilypond, a system for automated music engraving. In: XIV COLLOQUIUM ON MUSICAL INFORMATICS, 2003, Firenze, Italy. Pro- ceedings of the XIV Colloquium on Musical Informatics (XIV CIM 2003). Tempo Reale, 2003, p.167-172.

PARENTE, Peter. Audio enriched links: web page previews for blind users. In: INTERNA- TIONAL ACM SIGACCESS CONFERENCE ON COMPUTERS AND ACCESSIBILITY, 6, 2004, Atlanta, GA, USA. Proceedings of the 6th International ACM SIGACCESS Conference on Compu- ters and Accessibility. New York, NY, USA, ACM, 2004, p.2-8.

PETRUCCI, Lori Stefano; HARTH, Eric; ROTH, Patrick; ASSIMACOPOULOS, Andre; PUN, Thierry. Websound: A generic web sonification tool, and its application to an auditory web bro- wser for blind and visually impaired users. In: INTERNATIONAL CONFERENCE ON AUDI- TORY DISPLAY, 6, 2000, Atlanta, Georgia. Proceedings of the 6th International Conference on Auditory Display (ICAD 2000). Georgia Institute of Technology, 2000, p.6-9.

QUINN, Marty. Research set to music: The climate symphony and other sonifications of ice co- re, radar, DNA, seismic and solar wind data. In: INTERNATIONAL CONFERENCE ON AUDI- TORY DISPLAY, 7, 2001, Espoo, Finland. Proceedings of the 7th International Conference on Au- ditory Display. Georgia Institute of Technology, 2001.

QUINN, Marty; QUINN, Wendy; HATCHER, Ben. For those who died: A 9/11 tribute. In: IN- TERNATIONAL CONFERENCE ON AUDITORY DISPLAY, 9, 2003, Boston, Massachusetts. Proceedings of the 9th International Conference on Auditory Display. Georgia Institute of Tech- nology, 2003.

ROBERTS, Charles; KUCHERA-MORIN, JoAnn. Gibber: Live coding audio in the browser. In: Proceedings of the 2012 International Computer Music Conference. Computer Music Association, 2012, p. 64-69.

BODO, P. P. R.; SCHIAVONI, L. F. Web Sonification with Synesthesia Tools Revista Música Hodie, Goiânia, V.18 - n.1, 2018, p. 74-91.

ROGERS, Chris. W3C Web Audio API. Available on <https://www.w3.org/TR/webaudio/>. Ac- cessed: 2017-07-01.

SEEVINCK, Jennifer. Emergence in Interactive Art. Switzerland: Springer International Pu- blishing, 2017. 182p.

SCHAUERTE, Boris; WÖRTWEIN, Torsten; STIEFELHAGEN, Rainer. A Web-based Platform for Interactive Image Sonification. In: HUMANS AND COMPUTERS WORKSHOP, 2015. Pro- ceedings of the 2015 Humans and Computers Workshop. De Gruyter, 2015, p. 399-403.

WALSHAW, Chris. The ABC Music Notation. Available on <http://abcnotation.com>. Accessed: 2017-07-01.

WEINTRAUB, Annette. Art on the Web, the Web as art. Communications of the ACM. New York, NY, USA, Volume 40, Issue 10, p.97-102, October 1997.

WINBERG, Fredrik; HELLSTROM, Sten. Qualitative aspects of auditory direct manipulation: A case study of the Towers of Hanoi. In: INTERNATIONAL CONFERENCE ON AUDITORY DIS- PLAY, 7, 2001, Espoo, Finland. Proceedings of the 7th International Conference on Auditory Dis- play. Georgia Institute of Technology, 2001.

WYSE, Lonce; SUBRAMANIAN, Srikumar. The viability of the web browser as a computer mu- sic platform. Computer Music Journal, Cambridge, MA, USA, Volume 37, Issue 4, p.10-23, 2013.