Top of Page

Links to move inside this page.
  1. HOME
  2. Technology
  3. IIJ Technology (archives)
  4. HTML5 Next-Generation Application Platforms

HTML5 Next-Generation Application Platforms

October 13, 2011
(Original Japanese article translated on December 12, 2012)

Web browsers have rapidly become more and more sophisticated in recent years. In the years to come many native applications are expected to be replaced by applications that use HTML5 technology in conjunction with cloud services.
Here we will provide an overview of HTML5 and a technical explanation of the application that received top prize in the ASCII Art category of the "0th HTML5 Programming & Creative Contest".

Introduction

Web technologies are currently the most widely used on the Internet, and one of their core specifications is HTML4.01, which was standardized in 1999.
Because this specification was not revised for a long period of time, it gradually became outdated. So in 2004, a group composed chiefly of browser vendors formed "WHATWG," and began discussing extensions to the HTML specification. Following this, an HTML5 working group was formally established by the W3C (World Wide Web Consortium) in 2006, and the working draft of HTML5 was at last call in 2011. Work towards getting HTML5 to Recommendation status by 2014 is currently underway.

HTML5 Features

HTML5 specifications are being revised based on the following three points.

  • Improved compatibility
  • Definition of content meaning (semantics)
  • Development of more advanced application APIs

However, browser applications use a broader range of technologies than those defined in the W3C's HTML5 specifications, so specifications for related technologies such as CSS3/SVG/WebGL are also sometimes referred to as HTML5 (in a broad sense). In this article the term HTML5 is used to include these related technologies.

The reasons why people pay attention to HTML5 lately are listed as follows..

  • Enhanced browser performance
  • Widely deployed cloud services
  • Rapid growth of mobile devices

Enhanced browser performance has improved the speed of JavaScript, which had been perceived as operating slowly in the past, and has also made applications run smoothly in browsers.
Widely deployed cloud services have enabled the use of a variety of applications over networks, and HTML5 technology has begun to be used in the front-ends for social networking services.
Because of the rapid growth of smartphones recently, applications on the smartphone need to support both iPhone and Android. HTML5 applications in the browser work on both platforms and attract a lot of attention. It is said that many native or Flash applications will gradually be replaced by "cloud + HTML5" applications in the near future.
However, HTML5 is not a silver bullet, as its implementation is still patchy depending on the device, OS, or browser. It also cannot fully utilize the built-in functionality of the hardware and OS because it is sandboxed within browsers. This demonstrates that HTML5 adoption is still underway.
HTML5 and related technologies have an extremely wide scope, and are evolving rapidly. The fast release cycles of Firefox and Chrome indicate that browser vendors are implementing one new feature after another without waiting for standardization.
To demonstrate the new features of HTML5, we will examine the "HTML5 Radio" application that was produced using experimental functions before HTML5 standardization.

The Pioneering HTML5 Application "HTML5 Radio"

"HTML5 Radio" (note: runs on FireFox 4 or later, or Chrome 14 or later)blank

This application received top prize in the ASCII Art category of the "0th HTML5 Programming & Creative Contest".
Most HTML5 applications have extremely rich graphics, but this contest has an ASCII Art category that compels us to use poor graphics, so rich audio functionalities are included in the application instead.
One of the multimedia features in HTML5 is the ability to play audio and video without using plug-ins such as Flash. However, it had only supported basic functions such as play or stop, meaning it had not been possible to analyze audio data and use it within an application, or generate audio data using an application before these new audio features were introduced.
In this application, experimental APIs for handling audio data were implemented, such as the Audio Data API in Firefox (available in Firefox 4 and later) and the Web Audio API in Chrome (available in Chrome 14 or later).

"HTML5 Radio" uses both of these APIs to implement rich audio functions via primitive ASCII art graphics.

HTML5 Radio Functions

The functions of "HTML5 Radio" can be divided into the following two broad categories.

Control interfaces for audio functions using jQuery/jQuery-ui

Animation

  • Animation of musical notes dancing to the music (recursive processing using CSS3 transitions)
  • Display of the Fourier spectrum of music data on a level meter (scheduled processing using setInterval)

For the CSS3 animation of musical notes, new musical note symbols are generated when a beat is detected with a beat counter library.

Audio Data API / Web Audio API

Here we will explain technical details in the core audio data API rather than those in the animations.

Two APIs are currently implemented for audio data -- the Audio Data API and the Web Audio API. There is no compatibility between these APIs, and their features and usage are also completely different. For this reason, two distinct versions of "HTML5 Radio" were developed for Firefox and Chrome, and users can be redirected to the corresponding page after checking their browser agents.

The features of each API are listed below.

Audio Data API

Operating Environment FireFox 4 or later
Features Proper events are added to audio tag elements, and events are emitted in audio play, with audio data included in the event data. For output, data is written directly to the Audio object's methods.
Pros It has good compatibility with HTML5 audio tags, and can handle comparatively large pieces of audio data.
Cons It must be implemented using JavaScript even when conducting complex data analysis, etc.

Web Audio API

Operating Environment Chrome 14 or later
Features After a context to process audio data is created, audio files are loaded using Ajax, and processing is chained to nodes that apply a variety of audio effects. Finally, mixing is carried out before audio output.
Pros APIs have been implemented for a range of audio effect processing, making it possible to create advanced sound effects relatively quickly.
Cons Because audio data needs to be obtained via Ajax, it should be comparatively small. (In the specifications, MediaElementAudioSourceNode is provided for loading audio sources from audio or video elements, but it is not yet implemented in Chrome)

It could be said that the Audio Data API is a low-level API that enables raw audio data to be handled, while the Web Audio API is a high-level API designed for a variety of effects processing.

"HTML5 Radio" uses the APIs described above to carry out the following processing.

  • Playback, volume adjustment, and stopping of multiple pieces of audio data
  • Fourier analysis of audio data according to audio playback, and dynamic display of the results in a level meter
  • Creation and output of quasi-white noise when the radio is not tuned to a station

The figure below shows the implementation process for each of these.

Audio Data API / Web Audio API

As outlined above, it was possible to create the effect of playing music by tuning in to stations while producing noise like a real radio, even using APIs with completely different specifications. With the rapid spread of smartphones recently, games are another area where use of HTML5 is being highlighted. Advanced animation and sound effects are important aspects of games, so demand for APIs that carry out audio data processing is only likely to increase in the future.

In May 2011 the "Audio WG" working group dealing with audio data was established by the W3C, and the standardization process is underway. In addition to the Web Audio API mentioned above, this working group is also evaluating the technical specifications for the MediaStream Processing API that unifies processing for video and audio, such as the audio processing necessary for the capture or P2P communication of multimedia streams. In a few years it should be possible to carry out more advanced audio and video processing in any browser.

Reference Information

Audio Data API Specifications: https://wiki.mozilla.org/Audio_Data_APIblank

Web Audio API Specifications: https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.htmlblank

W3C Audio WG : http://www.w3.org/2011/audio/blank

Note:This article is based on information available as of October 2011, and details may differ from the time of writing due to subsequent changes to specifications or implementation status.

Shigeki Ohtsu

Author Profile

Shigeki Ohtsu

Application Service Department, Service Division, IIJ
Mr. Ohtsu joined IIJ in April 1999. After working as a systems integration engineer/consultant and service engineer, he is now focused on the testing and evaluation of new technology for applications, including HTML5, the NUI (Natural User Interaction) Kinect hack, and Node.js.

End of the page.

Top of Page