More Accurate Dialidol American Idol Statistics

UPDATE: Lots of people (well, more than 3) are finding this page by Googling for “dialidol.” You can find more current posts about DialIdol here.

Dialidol.com has some interesting statistics on how often a contestant’s phone number was busy during the voting, but the statistics look a little wonky to me. Every night they claim a 3% margin of error for their results, and then say that any contestant whose busy percentage is within 3% of the least-busy number could get voted off.

They’re free to say that, but it doesn’t make any sense statistically. Now, it’s been a long time since I had a statistics class, but I’m going to try to take a more statistically valid look at the numbers. This is probably overkill for an already biased sample set, but I don’t really want to be doing work right now, so I’m going to carry on.

Real Margin of Error

The margin of error should really be different for each contestent based on how many people tried to vote for that person with the Dialidol software. Last night, Ace Young had the fewest Dialidol voters, with only 372 vote attempts (compared to Elliott Yamin with 3202 vote attempts.) At a 95% confidence interval, that means that Ace’s results have a 5.1% margin of error. On the other side, Elliott’s results have a 1.7% margin of error.

If you bump it up to a 99% confidence level, then Ace’s results have a 6.7% margin of error, and Elliott’s have a 2.3% margin of error.

Full Results

These are all based on a 99% level of confidence.

And This Means What?

Well… not much. Let’s look at Ace, since he has the smallest sample size, and therefore the largest margin of error. The 6.7% margin of error (at the 99% confidence level) means there’s a 99% probability that his phone line was busy between 6.5% and 7.5% of the time (6.7% above and below his measured busy percentage of 7.0%).

For all of the contestants last night, there’s no case in which the rankings would be different than what Dialidol shows. Statistically, we can be 99% confident that each contestant is in the right spot on the list. The closest are Katharine and Elliott, since Katharine could be as high as 20.0% busy, and Elliott could be as low as 20.2% busy.

This is completely different than the way Dialidol is using margin of error. They have declared the results to have a 3% margin of error, and are saying that because of that, the actual busy percentage could be 3 percentage points lower or higher than where it was measured. In other words, Ace could be between 4% and 10%. This is using margin of error in a completely wrong way.

So, getting back to the right way. All this means is that there are enough people dialing to get a small enough margin of error as far as how busy a phone line was. But at the same time the data is skewed enough that it shouldn’t be a huge surprise if Lisa and Ace aren’t voted off tonight, for a number of reasons.

Biased Data

Let’s ignore the issue of not measuring text message votes, and just figure out how accurately Dialidol is measuring the busy signal percentages.

There are two big issues. One if that the modems can’t tell the difference between a “your call can not be completed” message, and a “thank you for voting message.” This means that the busiest lines are going to show more successfully completed votes than the less busy lines.

But the biggest issue is the calling traffic over time. Right when the show ends, there are busy signals all over the place. Eventually people stop voting, and the busy signals become fewer and fewer. Towards the end of the voting period, it’s much easier to get through than at the beginning. This is probably throwing off the measurements far more than the margin of error would take into account.

The problem is that the percentage of the time a phone line is busy changes dramatically over the two-hour voting period. It might be 95% busy at 10:00, right after the show ends, and 5% busy at midnight. So being 99% confident that Ace’s phone line was busy between 6.5% and 7.5% of the time just means that it was that busy when a Dialidol user was trying to vote for him. There’s no way of knowing what the temporal vote distribution looks like for each contestant.

The other factor that complicates things is that Dialidol has a “make these contestants safe” mode. You pick the contestants you like, and it votes for those that need it the most, based on the busy percentages so far. This has the potential of just amplifying any sampling errors. Let’s say that in the first 5 or 10 minutes, all the phone lines are pretty equally busy. I don’t know if that’s the case, but let’s assume it is. So the rankings that Dialidol comes up with are pretty random at that point.

Now the program will start to vote more for those people at the bottom of the list. It does this as the same time as the phone lines become less busy. The net result is that constestants at the bottom of the list would get more samples later in the voting period when the phone lines are less busy, so they’d just fall further to the bottom. Contestants at the top are presumed safe by Dialidol voters, get fewer votes later on, and so keep their high busy percentages.

Now, Ace all the way at the bottom with the least number of Dialidol voters would seem to contradict that theory. But then there’s Lisa all the way at the bottom, and with the third-highest number of Dialidol vote attempts. The second-highest number of vote attempts goes to melissa, who’s fourth from the bottom. That’s probably not enough to be statistically significant, but I’ll leave that as an exercise for the reader who actually remembers anything from statistics class (unlike me, I had to do some searching to remember how to calculate margins of error and all that.)

Summary

In summary, I sure am capable of typing a whole lot of boring and dry stuff about statistics. I can’t believe you read this far. Are you insane?

I guess the main thing to remember here is that Dialidol’s use of “margin of error” is pretty meaningless. The data is actually more statistically accurate than they claim, but there are other big issues that throw off the accuracy.

Personal Opinions

Just to throw these out there:

Taylor rocks.

Kellie is mind-blowingly stupid and not a very good singer, but does manage to crack me up (unintentionally) over and over, so I don’t mind her sticking around.

Chris. I usually like Chris, but last night was boring. I don’t know what the judges were going on about him “making the song his own.” Maybe they never heard the Red Hot Chili Peppers version, which Chris just did a bland version of?

Paris. My opinion of Paris was going down week after week, but last night was a big improvement.

Mandisa. Probably my least favorite performance of her’s, but that’s still well above most of the rest of the gang. I might give up on the season and stop watching if she gets voted off. I’m just barely hanging in now by skipping over the video clips and group songs and things like that.

Elliott. I didn’t like him last night, but I still think he’s one of the better singers they have.

Katharine. Same thing I said about Elliott, but much more fun to look at than Elliott.

Bucky. I’m still not sure how he’s made it this far.

Melissa. I like her sometimes, but don’t see her making it much further, and I’m ok with that.

Kevin. Alternatingly annoying and hilarious (again, unintentionally.) Maybe he should do a reality show with Kellie where we just laugh at what Kellie says and Kevin’s expressions.

Ace. Not bad. Not great.

Lisa. Some days I like her, some days I’m bored. I’m guessing Dialidol at least has her right, and she’s going home tonight.

Ok, that’s all folks.