Martinique's dashboard

Dumping raw sound data would be crazy indeed; Browsers all over the world would probably choke to death instantly. :-D

I don't believe it'd make sense to push more data for JS processing. Instead I think app developers should come up with ways to draw more creative and/or useful graphs based on the data that SM2 already provides. That's why I started this thread in the first place.

If you only need bigger, nicer waveform graphs, maybe some interpolation would help. – Martinique, on November 27, 2009 13:48

I solved this kind of case simply by building a player interface inside a position:absolute div (which can additionally be minimized/maximized to hide or show the play queue) and then placing a width:100%; height:100% iframe to cover the rest of the browser view. With a setup like that I can browse my music server and enqueue tracks with surprisingly simple code.

See the attached screenshot. Sorry about the crude looks; It's still very much in alpha state.

This would seem like a sensible approach, since I'm also one of those who'd love to use the waveform and eq data (and possibly some sort of streaming control with Flash 10 as discussed elsewhere) whenever possible, but also provide HTML 5 -based support for OGG Vorbis, which Flash still lacks.

How about a separate version (SM3/Pro?), with such extra features included but leaving any possible API limitations to be handled by application developers as they see fit? – Martinique, on October 29, 2009 08:43

Thanks, now the equation totally makes sense. Using Hz instead of the regular f got me a bit mixed up. I shouldn't have read your post before having any coffee.

... one of the things im doing is creating bands that match up with common frequencies used on graphic equalizers. the last band is the whole second half of the data =[

That's pretty much the same as what I did in my sample code (the updatePlayer function, where the height of each band is calculated from twice as many array indices as the previous one) and I've been quite satisfied with the results, but I'd like to take a look at your implementation as well, knowing you're a math geek :-D

Another thing I've begun wondering is if the band heights should represent the total energies of the frequency ranges instead of plain amplitude values. Higher f => higher energy (see this page for more info). That should certainly make the spectrum look less heavy on the bass end, but I have no idea if spectrum analyzers generally do that. Any thoughts on that?

Nice to hear that others too are curious about the eqData. I couldn't quite understand your index equation, though. The frequencies mentioned seem too small (maybe you meant kHz?). Could you please clarify it a bit?

The maximum values of 1.45 that you found confuse me even further. Another Flash-based audio player (can't quite remember which one) I tried produced a similar array, but with values between -1 and 1, according to docs, and with Math.abs they looked quite right. Adobe's documentation for computeSpectrum() in FFTMode is pretty non-existent, so it's not surprising that nobody seems to know anything specific about the spectrum data it produces. – Martinique, on October 21, 2009 06:35

Thanks a lot for the info. As I feared, Flash 8/9 obviously doesn't have enough "oomph" under the hood, but it's good to hear that 10 allows more freedom. And that idea behind as3mp3stream sounds absolutely brilliant! It'd not only take care of those pesky Shoutcast issues but would also end bandwidth problems like mine with its chunked loading, which, AFAIK, is exactly what desktop players do.

As I hinted earlier, the backend I'm developing currently relies on Range headers from the client when it comes to seeking and assumes that the client takes care of locating the next playable MP3/whatever frame within the data stream. WinAmp, for example, does a somewhat magical job estimating seek/playback position within a streamed MP3, even if it's VBR, although on the other hand it doesn't even show the seek bar when streaming OGG Vorbis (that input module of theirs apparently isn't equally advanced yet). Too bad that Flash isn't that "smart" yet.

It'd feel pretty wasteful to develop an additional method of seeking (queries, as suggested) for my server, when audio streaming can be performed satisfyingly with regular HTTP and a lightweight/plain-dumb backend. The advantage of avoiding backend-specific query strings would be that my web front-end would not only suit my own backend, but probably any server script that can serve M3U (which JS can easily download and parse) files containing the actual track URLs.

Limiting bandwidth artificially isn't really an option for me, because track URLs must be downloadable as well. What I've done to reduce bandwidth and RAM usage so far is to unload every track which hasn't loaded fully whenever that particular player instance is stopped and another one started, and do periodic playlist cleanups to unload tracks which haven't been played for a while.

Some have suggested a Java-based approach instead, but that'd feel too "20th century" IMO. :-D

Oh well, I guess I'll have to put my browser front-end on hold for a while and wait for a more suitable playback engine. I'll definitely keep on following these discussions, because SM is by far the most flexible playback option I've come across, and I'd really like to stick to it. Keep up the great work!

Many Flash video players seem to be able to seek and buffer on-demand (from the nearest video keyframe onwards, I assume), so it should be possible with audio as well, although I'm not familiar enough with Flash to say if it's got the necessary features under the hood.

In order to allow fast seeking, a video file must begin with a header describing keyframe positions in the file, but MP3 files don't have such information. What complicates things further with MP3 is a thing called "bit reservoir" (makes frames somewhat dependent on each other), which means that starting playback from an arbitrary MP3 frame is likely to cause decoding errors, but modern decoders are seemingly able to correct those. With newer formats such as Vorbis and AAC things are probably much easier.

Any comments from SM developers?

Now that I thought about it, instead of calculating averages, one could simply choose the maximum value of each array slice. Averages are OK with music-like signals, but they don't work quite right with test signals like single sine waves, although the FFT in Flash doesn't seem too accurate either. We'd need someone experienced to tell what's the best way. I know quite a lot about audio, but the exact mathematics behind spectrum analysis are out of my league. – Martinique, on September 01, 2009 19:58

I did some more testing based on your reply and found a reasonably quick way to make my eqData-based analyzer look more like an octave-band spectrum with 8 bands.

The first band represents the value of eqData[1] (I left 0 out because for some reason it didn't look right; it tended to indicate a low bass even when none was heard), the second one is an average of eqData[2] and eqData[3], the third band is the average of the next 4 values, and so on.

Thus for each band the frequency range is doubled, which should produce a logarithmic range, if my assumption of eqData linearity is correct. At least the frequency range seems to be constant as long as stretchFactor for computeSpectrum remains the same. Trying a value of 1 might be worth trying, though, since frequencies above 10000Hz are not really significant for our ears - that's assuming the highest frequency is currently 20000Hz as documented.

Here are some essential snippets from my code, in case someone wants to try my method:


// New array method to calculate sum of values:
Array.prototype.sum = function() {
for (var i=0,sum=0; i < this.length; sum+=this[i++]);
return sum;
};

// New array method to calculate average value of indices between b and e:
Array.prototype.sliceAvg = function(b,e) {
if (b==e) {return this[b];};
var s=this.slice(b,e);
return s.sum()/s.length;
};

// Number of analyzer bands (must be 8 with the algorithm used here):
var analyzerBands=8;

// How fast analyzer peaks fall (0=never, 1=instantly):
var analyzerFall=0.2;

// Place analyzer bars every n pixels:
var analyzerSpacing=10;

// Maximum height for analyzer bars:
var analyzerHeight=20;

// Array for object references (available after initialization):
var analyzerArr=new Array();

// Array for peak values:
var analyzerPeaks=new Array();

// Initialize analyzer graphics (when SM has loaded, for example):
function initAnalyzer() {
var a=document.getElementById('analyzer');
for (var x=0; x < analyzerBands; x++) {
analyzerArr[x]=document.createElement('div');
analyzerArr[x].style.left=(x*analyzerSpacing)+'px';
a.appendChild(analyzerArr[x]);
analyzerPeaks[x]=0;
};
};

// The analyzer bars could be styled somewhat like this:
// #analyzer div {position: absolute: bottom: 0; height: 0; overflow: hidden; width: 9px; background: #666666}

// This is called in whilePlaying:
function updatePlayer() {
var n=0;
for (var x=0; x < analyzerBands; x++) {
n=Math.pow(2,x)-1;
// Since Flash/SM seems to give values up to square root of 2 (~1.4),
// divide values by that to get a range of 0-1. Additionally, Math.sqrt
// "boosts" low values to make them visible in a small graph like this:
analyzerArr[x].style.height = Math.round(Math.sqrt(analyzerPeaks[x]=Math.max(this.eqData.sliceAvg(n+1,n*2+1)/Math.SQRT2,analyzerPeaks[x]))*analyzerHeight)+'px';
// Slow down bar fall off:
analyzerPeaks[x]-=analyzerFall;
};
};


I hope no serious typos slipped in while slicing code for the above example. At least it should give some idea how to cook a natural-looking logarithmic analyzer.

It's indeed an odd one, and I'd like someone to confirm if this happens on all OS X systems, but since it's so easy to work around, I'm marking this as solved.